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Foreword 


Our efforts to achieve Sustainable Development Goal 4, “Ensure inclusive and equi- 
table quality education and promote lifelong learning opportunities for all”, bring 
to mind Lewis Carroll’s Alice in Wonderland “Jf you don’t know where you want to 
go, then it doesn't matter which path you take”. 

The education systems in Africa have identified the need for twenty-first century 
skills, embarked into new curricula such as Kenya’s Competency Based Curriculum, 
and are introducing new teaching and learning methods such as inquiry-based learn- 
ing. However, like Alice in Wonderland, our sense of where we want to go with 
these changes is vague and undefined. On the one hand, there is a lack of unanimity 
on what success looks like, and on the other hand, we do not yet have effective 
instruments for measuring progress and outcomes. 

This volume engages primarily with the second of these issues. It describes an 
endeavour to identify where we are, relative to where we want to go. Reflecting 
priorities in East Africa, insights are provided into how cultural perspectives about 
twenty-first century skills can act as aids to their measurement, in the face of con- 
tinuing global differences in the understanding of the nature and identity of these 
competencies. The volume describes the Assessment of Life Skills and Values 
(ALIVE) initiative in East Africa which has engaged in a research-based endeavour 
to develop a contextualised set of tools to measure life skills and values through 
household-based assessment. 

ALIVE seeks to generate evidence on life skills and values in the East African 
context, develop and provide an open-source contextualised measurement tool, use 
the evidence to inform policy and drive public conversations, and use the tool devel- 
opment process as a pathway towards building and nurturing a community of prac- 
tice on learning assessments. This volume describes the achievement of these goals, 
drawing on the expertise and experience of the ALiVE team across Kenya, Tanzania, 
and Uganda. 

The volume addresses the problem of how to measure valued skills that underpin 
both education and daily life in such a way that practical information about youth 
performance is provided. The goal is to elicit data that will guide the emerging ‘life 
skills’ curricula in countries in East Africa. In order to trace the method and process 
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for solving the problem, the volume follows two themes. One theme concerns the 
prominence of twenty-first century skills in education over recent years, how this 
has taken form in East Africa through a collaborative learning approach, and how 
this can inform policy through articulating the need for and uses of household-based 
assessment of life skills in advocating for change in socio-economic and education 
policy. The other theme explores the issue of ‘global’ versus local understandings 
and definitions of twenty-first century skills through the lens of specific competen- 
cies — such as collaboration, self-awareness and respect, and problem solving — and 
then builds on this through reporting on the development of practical tools which 
reflect the local perspectives. This theme engages with the technical challenges of 
assessment of the competencies, demonstrating how the local conceptualisations 
lead to use of grounded daily life scenarios upon which to develop assessment items. 

This is a pathbreaking work in at least four ways. 

First, it recognises that the transition to twenty-first century skills requires inte- 
gration into existing curricula of more complex cognitive and affective competen- 
cies, such as critical thinking, innovation, collaboration, problem solving, 
self-awareness, and communication. While there is broad agreement and under- 
standing of how to measure skills such as numeracy and literacy, there is lack of 
clarity on the assessment and pedagogical practices that enhance some twenty- 
first century skills, and in particular affective skills. As a result, professional devel- 
opment programs for teachers do not provide guidance on how to nurture and assess 
competence in these character-based skills. This assessment provides an important 
model for assessing these skills, which can contribute to pedagogical insights. 

Second, the current tools used to measure twenty-first century skills are based on 
Western cultures and contexts and are sometimes not relevant to African cultural 
contexts. The ethnographic approach taken by ALIVE provides important insights 
into how collaboration, problem solving, and self-awareness skills are defined in 
sub-Saharan Africa, how they can be taught, and how they can be assessed in class 
and outside it. For example, the ALIVE contextualisation study showed that col- 
laboration is driven more by the need to stick together than by the goal to be 
achieved, and it reveals a fundamental difference in the understanding of the con- 
cept of collaboration which is not reflected in international assessments. 

Three, by using household-based learning assessments, ALIVE prioritises the 
important contribution that learning and skill development makes both in school 
and outside it. It recognises that assessments need to capture a wide range of learn- 
ers and attend to the learning needs of diverse populations. The information pre- 
sented in this volume should trigger further study to identify key differences in 
skills and competencies between in-school and out-of-school youth to enable suit- 
able teaching and training approaches. 

Four, ALiVE provides an important pathway towards nurturing a regional com- 
munity of practice on learning assessments and to generating the much needed 
knowledge on how to inculcate and measure life skills in the region. The psycho- 
metric value and rigour of the assessment tool, and assessed proficiencies of over 
45,000 adolescents and the learning achieved by those who participated, 
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demonstrates how collaborative local action can achieve unique and profound 
insights that cannot be generated by external researchers. 

This volume has taken the first step in that classical journey of a thousand miles 
that starts with the first step. It has provided an important framework for educators 
to contextualise the meaning and implications of skills such as collaboration and 
self-awareness. It should stimulate a continuing journey towards developing tools 
that are psychometrically rigorous, pedagogically useful, and context and culturally 
relevant. 


June 8, 2023 Ruth Kagia 


Ruth Kagia served as a Senior Adviser to the President of Kenya, as well as Head of the 
Presidential Policy and Strategy Unit between 2014 and 2022. Prior to this and from 1990 onward, 
Ruth served at the World Bank, including as World Bank Global Director for Education. In this 
capacity she led the establishment of the Global Partnership for Education (at that time known as 
the Fast-Track Initiative), currently the largest global fund dedicated to transforming education in 
lower-income countries. In 2008-2012, she served as World Bank Country Director for Southern 
Africa. Before joining the World Bank, Mrs. Kagia worked with the Kenya Government in teach- 
ing, education management, and research. She holds degrees from the University of Nairobi in 
Kenya, Harvard Graduate School of Education in the USA, and the Commonwealth University, UK. 


Preface 


This is no ordinary book. It is a mosaic of the lives of 20 East African authors, edu- 
cation and life skills enthusiasts and a few friends converging along the path of 
self-empowerment. It is the quest of East Africans to tell their own story, a story of 
learning through doing to not only develop and use contextualised assessments of 
life skills and values, but also figure out what it all means and why it matters. 

We are led by the vision of a generation of learners who have acquired the needed 
competencies to navigate the complexities of twenty-first century learning, living, 
and working. The Assessment of Life Skills and Values in East Africa (ALIVE) is a 
journey of hope. When we started in 2020, there were only a handful of assessments 
of these competencies in Africa. Even then, most of the assessment tools used were 
standardized questionnaires, developed with a Western lens and difficult to apply to 
our context. Most were self- and peer-rating scales and were inaccessible at local 
level as none were open-sourced. We hardly found anyone in East Africa with a 
reasonable level of expertise to develop these assessments. In the face of these 
obstacles, we decided to act. We started ALIVE as a collective impact and learning 
initiative targeting three outcomes — learn and develop a contextualised assessment 
tool to conduct a large-scale assessment, use evidence for public awareness and 
policy awareness, and establish a community of learning that raises the voices of 
East Africans about these competences. 

This dream started coming to fruition with the founding of the Regional Education 
Learning Initiative (RELD in 2017, a platform designed and dedicated to ensuring 
equitable access to quality education for East African children. A group of 20 civil 
society organizations from Kenya, Tanzania, and Uganda signed into a thematic 
group bearing the name Values and Life Skills (VaLi). Our initial meetings focused 
on tracing and learning from existing assessments and interventions globally. A 
meeting held in June 2018 in Dar es Salaam gave strong impetus to our burgeoning 
movement. Facilitated by experts who included Esther Care and Helyn Kim from 
the Brookings Institution, Ann Herschkowitz from the Education Development 
Centre, and Carina Omoeva from FHI 360, this meeting yielded our clarity on the 
three learning questions that have driven VaLi’s learning agenda ever since. We 
could finally see the light at the end of the tunnel. First is the clarification of what 
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these competencies mean in the East African context, and which of these should be 
prioritized and why. Second is the question of what works in nurturing these com- 
petencies in context. Third is which measurement approaches are needed and effec- 
tive to assess these competencies in East Africa. 

By August 2020, we had convened more than 10 learning and co-creation meet- 
ings, and three funders provided their support to kick-off this ambition. The self- 
empowerment journey was now in full swing, learning how to collaborate and how 
to develop a contextualised assessment tool. Consensus was reached among the 
VaLi members across the three countries, to start with three skills and one value: 
collaboration, problem solving, self-awareness, and respect. These were among the 
competencies prioritised by the curricula in the three countries, and what members 
felt would best represent the key domains of twenty-first century competencies — 
mostly the cognitive, the self, and the social domains. 

Unique to ALIVE tool development was the choice to balance between process 
and product. First, a contextualisation study was conducted in 15 districts across the 
three countries, covering a wide range of cultural and economic contexts. This qual- 
itative study targeted the clarification of the constructs informed by adolescents, 
their parents and teachers, and other persons working with this age group. It also 
generated raw material for the assessment questions, items, and tasks. Second, a 
mapping process was applied to identify 47 East Africans who had the disposition 
or interest to learn how to develop the assessments. They were drawn from govern- 
ment institutions, university departments, civil society organizations doing this sort 
of work in the region, and a few teachers as well as parents. Professor Esther Care 
(then of University of Melbourne) was invited as the process facilitator, teacher, and 
mentor. It took 45 weeks through COVID, multiple workshops (sometimes virtual 
and other times blended, with Tanzanians mostly able to hold in-person meetings), 
to apply the contextualisation results and outcomes of global literature reviews to 
develop skill structures, skill definitions, and the assessment framework and gener- 
ate and pre-test the assessment items and tasks that formed the ALiVE tool. 

Over this same period, two other processes were underway. The ALiVE Learning 
Community was convening virtual learning sessions (LearnShops) to build clarity 
on various assessment aspects and building momentum for learning, while public 
awareness and advocacy activities spoke on the importance of holistic education 
that included life skills and values. 

A highlight in the journey was the household assessment conducted between 
April and August 2020 in 20 counties in Kenya, 45 districts in Tanzania, and 20 
districts in Uganda, reaching over 45,000 adolescents aged 13-17 years. We cele- 
brate this not just as the first assessment of this scale on the continent, but also the 
value drawn from direct interaction with parents and community members in the 
tens of thousands of households that we visited across East Africa. Moreover, close 
to 1000 teacher trainees (and some teachers) were involved as assessors, offering 
them hands-on opportunity in conducting this kind of assessment, a valuable skill 
for their future teaching endeavors. This assessment was not just about delivering 
data; learning, awareness, and advocacy were its core ingredients. 
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This book compiles the perspectives emerging from the assessment process and 
results, as well as the localised experiences generated from this initiative. Given that 
more than half of the authors here are practitioners and not scholars, the authoring 
process was itself a learning agenda. Prof Esther Care, Dr Mauro Giacomazzi, and 
I offered time to train on academic writing, support to conceptualise the chapters, 
and exposure to relevant literature to draw from. Hidden in this book is the precious 
treasure of self-trust and determination, mutual purpose, and collaborative work. 
The book is a gift to ourselves, a shared mission with our education systems, and a 
special dedication to the children of East Africa. We hope it will be useful to you as 
reader, and a welcome to join our efforts in achieving our vision of learners who 
have embodied these competencies and are thriving in school, in work, and in life. 


Nairobi, Kenya John Kabutha Mugo 
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Chapter 1 A 
Global to Regional: From Twenty-First gies 
Century Skills to Life Skills in East Africa 


Esther Care © 


Abstract Education systems worldwide are adapting to demands from civil society 
and the workforce to better equip young people to function effectively in the twenty- 
first century world. The lag from awareness to aspiration to policy and to practice 
requires communities to contribute to building knowledge, developing tools, and 
representing society’s needs to government. Three countries in sub-Saharan Africa, 
Kenya, Tanzania, and Uganda, are benefiting from the efforts of a network of civil 
organisations working together with academia and government, which seek to 
enhance education provision. This introductory chapter establishes the context in 
which the Assessment of Life Skills and Values in East Africa initiative has devel- 
oped tools to measure adolescents’ proficiencies, and in so doing developed exper- 
tise in the assessment of life skills and values through a regional initiative. 


1.1 Introduction 


The Assessment of Life Skills and Values in East Africa (ALIVE) is an initiative of 
the Values and Life Skills (VaLi) thematic group of RELI, the Regional Education 
Learning Initiative, which latter comprises over 70 East African organisations, with 
the goal of ensuring learning for all. ALiVE’s goals as of date of writing were to 
advocate for education through the following objectives: 


e Develop context-relevant, open-source tools for assessing life skills in East Africa; 

e Undertake a household assessment targeting adolescent aged 13 to 17 years, both in 
and out of school (generate the evidence); 

e Use the evidence to draw attention to and increase awareness on the worth of these 
competencies among stakeholders (public policy advocacy); 
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e Elevate RELI-VaLi to a regional community of practice on methods and measure- 
ment of life skills, replicable at the national and regional levels for sustainability 
(transnational alliance building); 

e Enhance peer learning and feedback among the RELI member organizations work- 
ing on improving learning outcomes in East Africa (learning, sharing & capacity 
strengthening.) https://reliafrica.org/alive/ 

From 2020 onward, ALiVE, through its 47 team members, has undertaken studies 
to contextualise understandings of life skills and values, developed tools to measure 
these, and assessed over 45,000 adolescents through household-based assessment. 
ALiVE’s approach has been borne from serious scholarship and research, the expe- 
rience of its members in household-based assessment through Uwezo (a survey of 
children’s literacy and numeracy competencies), and its learning journey in assess- 
ment. Between April and August 2022, adolescents were assessed across a variety 
of competencies, including life skills, a value, digital literacy, and literacy, in Kenya 
(17,276), Tanzania mainland (14,645), Tanzania Zanzibar (2447), and Uganda 
(11,074). Note that Zanzibar’s education system is distinct from that of 
Tanzania mainland, and so participated in ALIVE as a fourth educational system. 

The output of these processes are the data that will inform ALIVE, its partners, 
and the ministries of education in the three countries about the proficiencies of 
13-17 year olds. Kenya, Tanzania, and Uganda have all re-visited their curricula in 
recent years in order to meet the learning goals that their governments espouse. 
These data provide another tool, usable by ministries of education, to fine tune cur- 
ricula, teaching strategies, and the enabling educational environments. 

An outcome of these processes is an increasingly skilled professional workforce, 
which has the capacity to translate learning goals into assessment tools, instruc- 
tional strategies, and policy guidance. 

The ‘great divide’, as signified by terms such as the ‘global south’ is bearing 
unexpected fruit. As developing economies take charge of their educational direc- 
tions, they increasingly self-reference. One symptom of this shift lies in research 
and practice that adopts a ‘contextualisation’ approach. The approach, epitomised 
by the ALIVE initiative, does not seek the substantive guidance of aid and develop- 
ment providers from high income economies, notwithstanding a U.S. centric per- 
spective that the latter are responsible for empowering and enabling ‘locally led 
development’ (Ingram, 2022). 

Another symptom of this shift is the adoption of educational directions consis- 
tent with local analysis of needs, rather than accepting the educational imperatives 
of aid providers. For example, ALiVE’s exploration of life skills and values repre- 
sents a perspective on education that is distinctly different from that of some global 
agencies that literacy and numeracy are what constitutes ‘learning’ and accounts for 
‘learning poverty’ meaning those unable to read and understand a simple story 
(World Bank et al., 2022) in low- and middle-income economies. The information 
about importance of a broader set of skills dates back to at least 2001 (e.g., Heckman 
& Rubinstein, 2001), to the effect that ‘non-cognitive’ skills contribute to determi- 
nation not only of educational achievement, but also income. That OECD’s 
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‘Programme of International School Assessment’ (PISA) has included competen- 
cies beyond language, mathematics and science in all rounds since 2003, is further 
testimony to the acknowledgement by OECD members that education indeed 
requires more than these foundational competencies. Although student attitudes, 
beliefs, and behaviours such as self-regulation and learning strategies were origi- 
nally captured within PISA from student questionnaires, the OECD has moved 
increasingly into measurement of ‘transferable’ or broader skills such as problem 
solving, collaborative and creative problem solving, and global competence. 
Reflecting on these trends, there is strong evidence of countries ‘mainstreaming’ 
many of these competencies within their curricula through areas such as global citi- 
zenship education (Benavot & Williams, 2023; UNESCO, 2021) notwithstanding 
little evidence of actual teaching and learning of related knowledge and skills. 

The achievement of the ALiVE initiative does not lie simply in the fact that over 
45,000 adolescents across Kenya, Tanzania, and Uganda have been assessed, but in 
the manner of its undertaking. Key features of this undertaking are: 


e That a loose consortium of education stakeholders, primarily from non- 
government not for profit organisations, but including government and the higher 
education sector, developed the assessment program in its entirety; 

e That the assessments were undertaken at household level in rural and urban 
areas, across isolated and in inner city areas; 

e That tools to assess three life skills and one value were developed from ground 
up by a collaborative group; and 

e That the tools were found to have sound psychometric qualities. 


Each of these features signifies a major shift in the coordination and directing of aid 
and development activities. The enterprise was designed and led by in-country pro- 
fessionals, whose intent was to gather large scale data on life skills in order to influ- 
ence policy, and to learn by doing. 


1.1.1 What Are These Competencies? 


The ‘21st century skills’ shift has gained pace in many national education systems 
over the past decade. Notwithstanding its ubiquity, there remains confusion about 
what exactly this term means, and what competencies and characteristics gather 
under its umbrella. A review of terminology that was in use around 2016 to describe 
twenty-first century skills included more than 50 terms. The review was part of an 
initiative which was then referred to as Skills for a Changing World (Care et al., 
2017). The fact that such a review was even necessary reflects the confusion gener- 
ated by the plethora of human characteristics and competencies that educators, 
employers, governments, and global institutions regard as essential for today’s 
young people. Table 1.1 presents a re-interpretation of some of the review’s more 
common terms. 
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Table 1.1 Some common terms associated with ‘breadth of skills’ 


Category Terms Category | Terms 
Temporal Twenty-first century skills | Process Learning skills 
Future world skills Life education skills 


Future ready skills 
Permanent skills 


4 


Outcome Work skills Relationship Dynamic skills 

Skills to succeed in life Connector skills 

Skills for social progress Transferable skills 

Life skills | | Transversal competencies 
Priority/importance | Accelerator skills Counterweight | Soft skills 

Essential life skills Noncognitive skills 

Core skills Integral skills 

Critical skills Generic capabilities 


| Foundational skills 


The categorising descriptors provide a perspective somewhat different than the 
substantive classifications (such as social-emotional, cognitive, affective, etc.) that 
are frequently used. The identification of ‘temporal’ in particular is interesting 
because it self-consciously acknowledges the criticism often levelled at the ‘21st 
century skills’ movement—that this term implies relevance only to this current cen- 
tury, whereas the competencies have clearly been relevant across centuries. 
‘Outcome’ is a clear reference to the functionality of skills in daily societal life. The 
‘priority/importance’ category is also self-evident, in that these terms claim for pre- 
eminence of the skills. ‘Process’ skills link nicely to the education space as well a 
functioning in daily life. The ‘relationship’ category is one which has high visibility 
in academic research (particularly in discussion of problem solving) and in some 
regions (e.g. South East Asia). The category includes several terms which empha- 
sise the generalisable and adaptive nature of the skills. A final category that illumi- 
nates discussion about terminology is ‘counterweight’. The category explicitly 
identifies what these skills are considered ‘not’ to be, and it is notable that some of 
its terms can carry negative connotations. 

Table 1.1 draws predominantly on use of the term ‘skills’, but it is clear from 
incorporation of many of these proficiencies now included in curriculum frame- 
works world-wide that this is not the only, nor necessarily the preferred, term. For 
example, the Australian curriculum refers to such proficiencies as ‘general capabili- 
ties’, Kenya as “core competencies’, Uganda as ‘generic skills’, Singapore as ‘21st 
century competencies’, and Zambia as ‘competences’ (and values). There are natu- 
rally many other terms in languages other than English. This chapter adopts the term 
“generic competencies’ when appropriate to do so. 

The ALIVE initiative adopted the terminology of ‘life skills’ and ‘values’. This 
terminology presents some difficulties for communication, both within East Africa 
(Joynes et al., 2019) and more widely (Global Partnership for Education, 2020). The 
term ‘life skills’ has frequently been used in Africa over the past decade to describe 
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technical-vocational skills, and also by the aid and development sector and by min- 
istries of education. In the latter case, its use has frequently been in the context of 
acquisition of knowledge and skills to deal with problems, most visibly associated 
with education about HIV AIDS, and drug and substance abuse, as well as with 
strategies to protect oneself from abuse by others (Akyeampong, 2014). This asso- 
ciation may pose a communications and dissemination risk to perceptions of the 
competencies as holistic and adaptive, rather than as protective factors alone. 

Classifications of the human characteristics and competencies that populate a 
‘breadth of skills’ are multitudinous and varied; and there is no one classification 
that has been accepted universally by national education systems, uni- and multi- 
lateral organisations dedicated to improvement of education provision, or work- 
force analysts and employers. Over the past three decades, classifications have 
varied from the simple to the complex. For example, we see the cognitive versus 
non-cognitive dichotomy promoted by Heckman and colleagues (e.g., Kautz et al., 
2014); to the Knowledge, Skills, Attitudes, Values and Ethics (KSAVE: Binkley 
et al., 2012) framework; through to the nature of global competence including 
knowledge, skills, attitudes and values (OECD, 2018), and on to global citizenship 
across cognitive, social, and behavioural functions (UNESCO, 2019). 

Seminal writings that have influenced the field have drawn on concepts of citi- 
zenship and society (Delors, 1996); on key competencies (OECD, 2001; Rychen & 
Salganik, 2003) including tools for interacting effectively, engaging with others, 
taking responsibility for managing self within the broader social context; and on 
‘deeper learning’ (Pellegrino & Hilton, 2012). More recently, the Sustainable 
Development Goals (SDGs; UNESCO, 2018) have made explicit not only the valu- 
ing of literacy and numeracy but also the need to equip citizens with technical and 
vocational skills, social and cognitive competencies, and the characteristics associ- 
ated with global citizenship. The SDGs signal an emphasis on the breadth of skills 
necessary to prepare children, youth and adults comprehensively for twenty-first 
century citizenship and life. 

There have been few efforts to identify classifications that are idiosyncratic to 
regions. Among these few are contributions from East Asia (Cheng, 2017) and 
South East Asia (UNESCO, 2015). In Africa, there have been few attempts to sum- 
marise frameworks of these competencies. One output of the Learning Metrics Task 
Force (LMTF, 2013) was identification of seven domains of learning, including the 
learning approaches and cognition domain, and the social and emotional domain. 
This was a global rather than regional enterprise but included a significant number 
of African countries. Another initiative that sought to map competencies being tar- 
geted by nine sub-Saharan countries that were members of TALENT, based in 
UNESCO, Dakar (Kim & Care, 2020), found core skills rather than common frame- 
works across their curricular goals — such as creativity, initiative, problem solving, 
critical thinking, collaboration, and communication. This cluster of skills is similar 
to that found from a larger scale mapping of over 160 countries (Care & Kim, 2018). 
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1.1.2 Why These Competencies in the Curriculum? 


As national education systems seek to broaden their curricula and teaching, these 
are responses to both global and local imperatives. The global we see reflected in 
much of the literature cited in the introductory sections of this chapter. The local 
imperatives can be seen reflected in individual country vision statements, which 
vary from responses to economic imperatives, citizenship, values, and religious 
concerns, and aspirations for development of the individual. Accordingly, the ratio- 
nale for ‘skills’ prioritises responses to a changing world socio-economically, con- 
cerns about the attitudes and values of youth, discontent with current education 
provision and student learning, and awareness of environmental and sustainabil- 
ity issues. 

Year by year increasing numbers of children attend formal education. As the sec- 
tor becomes more inclusive, so it must be more adaptive in order to meet the learn- 
ing needs of more diverse populations, and more diversified itself in order to meet 
the socio-economic needs of the state. However, the universalisation of education 
provision has not been accompanied by universalisation of expected outcomes. The 
existing concerns about slowness of progress toward the 2030 targets (UNESCO, 
2018) have been exacerbated by the repercussions of the COVID-19 pandemic, dur- 
ing which more than 190 countries implemented nationwide closures, hypothesised 
to exacerbate existing inequities and obstruct educational progress toward the SDG 
targets (UNESCO et al., 2021). 

As the education sector has struggled to continue provision of teaching or teach- 
ing materials to youth over the past 3 years, planners trimmed down curriculum and 
developed materials specific to self-directed learning (e.g., Department of Education, 
2020; UNESCO et al., 2021). At the same time, for those students who were able to 
access learning materials at all, their motivation and capacity to direct and manage 
their own learning bring the matter of self-direction, independence, and problem 
solving to the fore. We wait for more insights about the relevance of generic compe- 
tencies for self-management of learning, particularly in low-income countries 
(Betthauser et al., 2023). 

In the face of continuing concerns about the directions and implementation of 
education provision world-wide, the visibility of curriculum reform stands as an 
acknowledgement of the need to broaden the nature of that provision. Such acknowl- 
edgements are an essential step which can stimulate wider discussion of new direc- 
tions, and lead to further reforms in pedagogy and in assessment. As documented in 
the case of several countries in South East Asia (UNESCO, 2016; Care & Luo, 
2016) curricular reform that incorporates generic competencies has typically not 
been accompanied by timely reform initiatives in teacher training, or in provision of 
aligned learning resources. This trend points to continued lack of acknowledgement 
of the need for systemic integration of the skills, requiring a major rethink of how 
teachers, particularly in the secondary sector, can incorporate the skills in their 
teaching—through modelling and through focus on process. As increasing numbers 
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of countries and regions begin the journey towards more holistic concepts of educa- 
tion, these learnings have the power to inform more successful shifts in these 
countries. 


1.2 ALiVE 


Given the wealth of information now available from other countries which have 
embarked on the ‘21st century skills’ or generic competencies path, Kenya, 
Tanzania, and Uganda are well-placed to take an informed approach to policy and 
practice. Alert to these issues, the ALiVE initiative has taken a grounded approach 
to the informing of policy which will prioritise alignment across the major actors of 
an education provision system—curriculum, assessment, pedagogy, and learning 
resources. 

The education systems across the three countries have each reviewed their cur- 
ricula in the past 5 years, and have included what they variously refer to as ‘core 
competencies’ (Kenya: Kenya Institute of Curriculum Development, 2019), “generic 
skills’ (Uganda: National Curriculum Development Centre, 2020), and ‘learning, 
life, and soft skills’ (Tanzania: Tanzania Institute of Education, 2019). The national 
perspectives and consultations within each country are reflected by the differences 
in how each conceptualises these competencies. For ALiVE, a key factor was to 
identify a common set of valued competencies across the three countries, in order to 
ensure that the information captured by the assessment tools would be relevant and 
useful. A high-level overview is provided in Table 1.2. 

Table 1.2 lists the values and life skills that are identified in the education sys- 
tems’ curricular frameworks or visions. Each system typically also highlights key 
generic competencies which draw on the focus values and skills as well as others. 
There are some terminology differences across the countries, which are highlighted 
as relevant. There is no doubt that the countries are espousing similar sentiments in 
terms of their hopes for their youth. These similarities made possible the ALIVE 
aspirations for development of one common set of tools to use to capture a picture 
of young people’s competencies. This ‘picture’ would then provide evidence on 
current functioning levels to support system, school, and teacher identification of 
‘zones of proximal development’ (Vygotsky, 1978) for instruction. 

Notwithstanding the clarity of the countries’ aspirations, each remains in early 
days of implementing what amounts to a shift in educational policy from focus on 
academic knowledge and skills alone. There are several factors that typically impact 
on implementation (UNESCO, 2016; Care & Luo, 2016). These include: 


e Confusion about what the competencies actually are, which are valuable, how to 
conceptualise, how to integrate into practice, and how to evaluate outcomes (due 
to lack of knowledge of how to deconstruct to identify contributing subskills, 
integrate into classroom teaching); 
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Table 1.2 Commonality of values and skills across the four educational jurisdictions 


Values 

Kenya 

Care and compassion, ethical, excellence, honesty and trustworthiness, respect, responsibility, 
trust, understanding and tolerance 

Tanzania 

Equity, inclusion, multilingualism, patriotism, quality, self-reliance, sustainability, transparency 
and accountability 


Uganda lower secondary 
Integrity and honesty, patriotism, peace and harmony, positive attitude toward work, respect for 
human rights, self-control 


Uganda upper secondary 

Creativity and innovativeness, hard work for self-reliance, honesty, integrity, justice and fairness 
in dealing with others, national unity, national consciousness and patriotism, respect for 
humanity and environment, social responsibility, social harmony 


Skills Kenya | Tanzania | Uganda 
Critical thinking x x x 
Creativity (and Imagination‘) (and Innovation") x X X 
Problem solving x x x 
Communication x X x 
Collaboration, Cooperation” x X X 
Learning to Learn“, Initiative", Learning/Self-directed Learning" x x X 
Digital literacy, Information Media and Technology, ICT x x x 
proficiency” 

Leadership, Negotiation, Flexibility, Decision-making, Productivity x 


Key. " Tanzania; " Uganda 

Note. Sources for the tabled information: Kenya Institute of Curriculum Development (2019), 
Tanzania Institute of Education, MoEST (2019), National Curriculum Development Centre, 
Uganda (2020, 2023) 


e A policy and then inputs focus on implementation (of which curricular integra- 
tion is a visible sign) without review of the necessary infrastructure (teacher 
training, revised texts or teaching materials) (Benavot & Williams, 2023; 
UNESCO, 2021); 

e Lack of community understanding of the relevance and functionality of these 
competencies, and therefore lack of community voice to stimulate action. 


1.2.1 Contextualisation 


A major dissonance exists in the formal education environment and individuals’ 
daily lives. Arguably, this dissonance is to blame for communities’ lack of under- 
standing of this particular education shift. In many countries, the primary identity of 
education lies in its provision of foundational skills in literacy and numeracy; and 
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secondarily in its role in equipping those who are fortunate by birth or their intrinsic 
qualities, to prosper beyond the reach of the majority. 

Demonstrating the relevance of knowledge, skills, attitudes, and values in daily 
life has been the path taken by ALIVE. To use the experiences of young people from 
one day to the next as the vehicle for demonstration of life skills and values, has 
been the strategy to link the two worlds, and to elicit information from one world to 
inform the other. This strategy has been made possible through the initiative’s focus 
on contextualisation—what makes sense to the individual and community through 
common perspectives on social, religious and cultural life and manners. In effect, 
contextualisation is about ensuring authenticity of concept and of practice. 

ALIVE has based the assembling of evidence to inform policy on ‘contextualisation’ 
(e.g., Giacomazzi, 2022) through exploring local understandings of socially defined 
characteristics and competencies (see Giacomazzi, 2024; Chap. 3, this volume). 
In turn, contextual understandings have informed the development of assessment 
tools, drawing on ‘authenticity’ perspectives (e.g., Care & Kim, 2018) from the 
field of assessment. The two concepts—contextualisation and authenticity—are 
complementary elements in ALiVE. Contextualisation provides the means to ensure 
that target conceptualisations of human characteristics and capabilities are mean- 
ingful to the participating populations, while striving for authenticity in develop- 
ment of situation-based and performance-based tasks provides a realistic 
environment in which to demonstrate proficiencies. So, the target life skills and 
values are contextualised, and the measures of these are developed from ‘authentic’ 
daily life (see this volume: Scoular & Otieno, 2024; Ngina et al., 2024; Care & 
Giacomazzi, 2024). 


1.2.2 Assessment 


Assessment is a much-maligned notion, often deservedly so. It can be used to limit 
opportunities rather than open doors. However, it has two benefits that cannot be 
under-estimated. Both of these benefit stem from the same characteristic—assess- 
ment provides information. That information can be used at medium to large scale 
level to guide education providers concerning what has been achieved or what is 
wanting. The information can also be used at individual and group level to guide 
learning, primarily through the methods popularised by Black and Wiliam (1998) as 
formative assessment. The benefits can only be realised where those responsible for 
the assessment are clear on the purpose of the assessment, the target characteristic 
or competency of that assessment, and the target individual, group, or population for 
that assessment. This simple concept is challenged in implementation. A particular 
challenge, encountered and welcomed by ALIVE, is that of being clear on the target 
characteristic or competency to be assessed, and how those characteristics or com- 
petencies will be expressed and recognised in the target populations. 
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1.2.3 Assessment and Its Authenticity 


A primary convention adopted by psychometricians and by users of assessments, is 
that the assessment must clearly target the construct of interest (whether capabili- 
ties, knowledge, characteristics), and will target this in a way that will be familiar to 
the individual being assessed. These are two forms of alignment. For example, if 
one wants to find out if a child can add two numbers less than 10, then a task can be 
set that will require exactly that type of calculation—this is the first alignment, that 
what is assessed is an indicator of the targeted construct. If a child has been famil- 
iarised with the concept of addition of two numbers less than 10 through the use of 
manipulable objects, then the task can be provided in that mode—whether physi- 
cally or in an online environment. If the child has been familiarised with the concept 
via the use of symbols, and/or use of pen and paper, this mode can be used—this is 
the second alignment, that how one is assessed is consistent with the manner in 
which one has learnt. Of course, the alignment is not always as clear as in this 
example, particularly as the target construct becomes more complex, and draws on 
several subskills or other contributing factors. 

Given this history, the assessment of ‘21st century skills’ is an interesting phe- 
nomenon. Often referred to as ‘difficult to measure’ capabilities, there is little doubt 
that these skills fall into the category of being complex, and drawing on multiple 
competencies. This may account to some degree for the ‘difficult to measure’ per- 
ception. Besides this of course, is that these competencies have not been included 
extensively in curriculum, in teaching, and therefore in assessment. Since there may 
be few examples of how such skills are integrated into daily classroom practice, 
their lack of visibility does not provide a recognisable way of assessing them. 

Much of the focus on generic competencies has been around their use in daily 
life and in work, and their hoped-for role in helping societies to identify a better 
world future. These—daily life, work life, and problem solving—are the ‘authentic’ 
playing fields for the skills, and many of the ways in which it is presumed they will 
be demonstrated, is through performance in situ—as distinct from stating what one 
has learnt as often the case in the education environment. In this distinction lies the 
assessment approach adopted by ALIVE, to elicit likely reactions to scenarios rather 
than for adolescents to self-rate or evaluate themselves (see Nansubuga et al., 2024; 
Chap. 7, this volume). 

The five dimensions for authentic assessment proposed by Gulikers et al. (2004) 
provide a useful framework through which to combine authenticity demands of 
contextualisation and those of assessment. 

The first concerns the assessment task. An authentic assessment is one that 
requires application of what an individual has learned to a new instance of a familiar 
situation or issue; it should be realistic, and requires some performance on the part 
of the respondent (Wiggins, 1998). The medium therefore provides for the demon- 
stration of proficiencies directly rather than providing self-referenced estimates of 
one’s abilities, or merely providing knowledge-based perspectives on an issue. The 
assessment requires transfer of learning, a characteristic synonymous with the 
ambitions associated with the learning and development of generic competencies. 
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This notion of authentic assessment is removed from the narrower view of authentic 
assessment by teachers which is focussed on including multiple forms of assess- 
ment in the classroom. For example, the use of varied assessment forms described 
by Darling-Hammond and Snyder (2000)—portfolio, exhibitions, case reports, and 
problem-based inquiry—does not engage directly with authenticity of assessment 
and its alignment with contextualised understandings of the target behaviours or 
competencies. This approach acknowledges the different backgrounds of students, 
but ‘solves’ the issue by recommending delivery flexibility, as opposed to looking 
more deeply at the underpinnings of curriculum and contextualised meaning of con- 
cepts. The remaining dimensions are all key into the matter of contextualisation. 
These include the physical context, the social, the assessment result or form, and the 
criteria for evaluating assessment performance. 

A precursor to the assessment event itself is the matter of ensuring that the target 
competency is understood within its ‘activation space’. In other words, the assess- 
ment event cannot be authentic if the target competency itself is not contextual. 
Accordingly, the physical and social context of the assessment is not unique to the 
event, but is a necessary component of the original contextualisation process. With 
this matter checked, then the next significant question for ALiVE concerned the 
authenticity of the assessment event. Authenticity in assessment is not guaranteed 
by the contextualisation process; also, to be considered is the assessment act, or 
event, itself, and how this is experienced by the adolescent respondent. Strongly 
linked with the assessment and its form is how an individual’s responses are 
evaluated. 


1.2.4 The Vexed Matter of Validity 


Does a measure provide information that is true to a competence of interest? The 
whole process of contextualisation, to ensure that the ‘labels’ for the life skills and 
values represent shared understandings of these; the technical process of developing 
assessment tools that are true to the competence, and true to the context; do these 
ensure that the data gathered are a true representation of the adolescents’ skills 
and values? 

ALIVE engaged in a large-scale household-based assessment program, querying 
adolescents about their reactions and responses to daily life issues. How frequently 
have teams of adults come to urban as well as far-flung communities to engage one 
to one with young people. What are their likely thoughts or interpretations of the 
event itself? How might these have influenced their responses? 

For the majority of adolescents, their only experience of adults asking a series of 
questions and noting down their responses, has been in the formality of a classroom, 
or for employment purposes. And that formality has been associated with the knowl- 
edge of the adolescent that they would pass or fail, do well or do badly, be selected 
or not. That formality has also been associated with expectations about the dynam- 
ics of oral interactions with adults. The ALiVE assessment process was one in 
which adolescents were invited to answer in a conversation style, to engage in the 


12 E. Care 


interaction in the informal environment of the home, to provide their opinions and 
likely reactions—a very different adolescent-adult interaction than likely experi- 
enced previously. 

The impact of the possibility that the interaction might have been experienced 
with discomfort or awkwardness, is not known. What is known, is that all adoles- 
cents were approached in the same way, with the same instructions, and the same 
positive affect. To this degree, therefore, the results across the adolescents can be 
seen as comparable. Whether those results are under-estimates, however, is not 
something that can be known in these early days of the assessments of life skills and 
values. It is possible that specific community environments could differentially 
influence adolescent response modes and quality of those responses. 

How these results might translate into performance that draws on the same com- 
petencies, but in a formal school environment, using curricular-based content, is 
another unknown. The promise of the outcomes of ALIVE is that the competencies 
can be measured, and that is an indicator of immense value for the education sys- 
tems in which these adolescents engage. 


1.3 The Narrative 


The following chapters provide the narrative that threads the ALiVE learning, prog- 
ress, activities, and outputs. The chapters are written by those who have been part of 
the years long processes from contextualisation studies through to reporting results 
in national and global forums. Shariff et al. (2024) provide a more detailed descrip- 
tion of ALIVE, explaining its motivating force in terms of labour force and educa- 
tional needs. The chapter details how the activities that comprise the initiative 
moved from engaging a large technical team through to the assessment of over 
45,000 adolescents in the participating countries. Moving on from the over-arching 
view, Giacomazzi (2024) sets the contextualisation stage upon which ALIVE sits, 
describing the pivotal issue of ensuring that what is being aspired to is relevant and 
understood in the moment and in the place. He describes how ALIVE set about the 
process of understanding the life skills and values in the forms that they are appreci- 
ated by the main stakeholders in ALIVE — the parents, the children, the educators. 
With the first three chapters having established perspectives on the target competen- 
cies, the shape of the initiative, and the focus on contextualisation, the next three 
chapters take us to targets of the assessment, the skill of problem solving (Care & 
Giacomazzi, 2024), the skill of self-awareness and value of respect (Ngina et al., 
2024), and the skill of collaboration (Scoular & Otieno, 2024). Each of these skills 
and value chapters provide details of the conceptual structures and assessment 
frameworks of the constructs, drawing attention to different characteristics of these 
across global literature and measurement within ALiVE. Some of these characteris- 
tics are then illustrated by drawing on the ALIVE tool, and describing the approaches 
taken to task development for the final survey. Distinctive features of ALiVE’s 
approach to assessment are discussed in the next three chapters. First, Nansubuga 
et al. (2024) highlight approaches adopted in sub-Saharan Africa for the 
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measurement of ‘21* century skills’ in order to paint a picture of mainstream 
approaches to measurement of these skills. Next, Nakabugo et al. (2024) draw atten- 
tion to the household-based mode of assessment used in the program. This and the 
following chapter by Mutweleli et al. (2024) draw attention to the ground-breaking 
nature of this assessment enterprise. Both the measurement of these generic compe- 
tencies and use of household-based approaches to assessment are relatively recent 
in their own right. Combining these two initiatives provides a proof of concept 
illustrating the viability both of capturing the skills and doing so in environments 
not dependent on manipulables or technologies. Muteleli and colleagues describe 
how and why a scenario-based approach to assessment was used for three of the 
target constructs — problem solving, respect and self-awareness. Having established 
the context, the constructs, and the methods, the narrative shifts to outcomes and to 
reflection. Ariapa et al. (2024) provide the technical details behind the methods of 
test and scale development used, and describe the proficiencies of adolescents 
across Kenya, Tanzania mainland, Uganda, and Zanzibar in practical terms—what 
the adolescents can do. They describe the way that tasks and their items were tagged 
to the constructs, give details about the psychometric qualities of the tool, and pro- 
vide survey level results that present the proficiency levels of the adolescents. The 
final two chapters turn to organisation and aspiration. Turner et al. (2024) draw the 
portrait of the collaborative endeavour across the three countries, their networks and 
organisations, and individuals, to seek understanding of how this multi-faceted 
technical project survived and thrived. And Mugo (2024) takes the narrative back to 
its beginning, focussed on global and local policies and the strategic impact of edu- 
cation shifts for Kenya, Tanzania, and Uganda. He explores the systemic issues 
associated with use of evidence to inform policy, reflects on ALiVE’s path, and 
draws five implications for policy and practice. 


1.4 Conclusion 


The goal of ALiVE is to inform the education visions of the three participating 
countries, by providing evidence of adolescents’ current functional levels associ- 
ated with life skills and values valued by their education systems. This goal repre- 
sents the understanding that the intentions of education systems and their consequent 
inputs are insufficient to bring about change. Additional evidence, in the form of 
achievement data gathered from the concerned adolescent population, must be 
brought to bear in order to identify exactly what the current functioning of those 
adolescents is. This information then provides a road map to develop that function- 
ing. The following chapters describe a technical achievement. The achievement lies 
in two phenomena. The first is the development of an assessment tool to measure 
selected life skills and value, which when administered at large-scale household 
level, generates robust evidence of how a representative sample of young Africans 
see themselves and others, how they can solve problems and collaborate. The sec- 
ond is the collaboration across a large group of individuals and organisations pri- 
marily from non-government sector but with commitment from government and 
academia, to engage in learning by doing. 
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Abstract Life skills are rapidly becoming recognised as critical skills for children 
to acquire in order to succeed in school and in life. This is particularly true in the 
sub-Saharan African (SSA) context where youth educational attainment is low, 
unemployment and informal employment is high and a growing youth bulge pro- 
vides both an opportunity and a high risk for the development or under development 
of SSA countries. Developing appropriate life skills and values in youth has become 
imperative for education systems in order to ensure youth have the chance to suc- 
ceed, especially in terms of employment after finishing school. Global large-scale 
learning assessments have primarily been focused on basic or foundational compe- 
tencies such as literacy and numeracy. These assessments have paved the way in 
showing how large-scale evidence generation can influence policy and practice in 
different countries. When it comes to the life skills and values space however, few 
assessments have produced evidence to influence systemic shifts in educational 
delivery. This chapter outlines how the Assessment of Life skills and Values in East 
Africa (ALIVE) collaboration has addressed gaps in large scale evidence of life 
skills and values, and more specifically a lack of life skills and values assessments 
contextual to East Africa. 
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2.1 Introduction 


Values and life skills are understood and defined differently in different contexts 
(Care, 2024, Joynes et al., 2019). A process initiated in East Africa by members of 
the Regional Education Learning Initiative (RELI) in 2018 explored the use of the 
different conceptualizations in Kenya, Tanzania, and Uganda. Umbrella terms such 
as twenty-first century skills, social-emotional learning competencies, transferable 
and transversal skills, soft skills, and whole child development competencies were 
explored. It was concluded that despite the perception of ‘life skills’ referring to the 
need to equip children with attitudes and behaviours to support the control of HIV- 
AIDS and substance use in several African contexts (Akyeampong, 2014), the term 
was also accepted in curriculum as referring to the more generic skills. Beyond the 
concept of skills was a set of personal attributes which constituted values. The RELI 
group then adopted life skills and values as the overall concept for use in their work 
in East Africa, and the Assessment of Life Skills and Values in East Africa (ALIVE) 
initiative was born. 

This chapter reviews the use of learning assessments to influence policy. The 
review provides a context for ALiVE’s goals, the challenges it is designed to address, 
and how it addresses them to ensure children in East Africa acquire life skills and 
values to thrive in school and in life. 

There is an increasingly identifiable link between these generic skills and work 
readiness that is considered urgent especially in sub-Saharan Africa (SSA). 
According to the Global Employment Trends for Youth 2022 report (International 
Labor Organization, 2022), youth unemployment in Africa is below the world aver- 
age for both men and women, with women at 13.3% and men at 12.3%, for a total 
average of 12.7% in 2022. The seemingly low levels of unemployment reflect the 
fact that most young people in SSA cannot afford to remain unemployed. What is 
often not reflected in the numbers is the quality of employment and the need to 
engage in insecure, often low-productivity, jobs for income (International Labor 
Organization, 2022). Many young people are engaged in informal jobs due to there 
being a lack of alternative options and little information and support with regard to 
other options. This is a cause for concern. The growing youth bulge presents an 
opportunity for future development of countries willing to tap into the so-called 
demographic dividend through investment in human capital. This investment would 
contribute to ensuring that young people develop the technical and socio-emotional 
skills and values necessary to become productive and contributing members of soci- 
ety (UK Aid, 2018). On the other hand, it is feared that if highly youthful societies 
do not take advantage of the demographic dividend, it may result in detrimental 
side-effects, exacerbating crime, violence, and continued poverty (UK Aid, 2018). 

Recent studies in employability conducted in East Africa are showing that more 
effort is needed to invest in education and skills building of youth in the region. A 
study on perceptions of employability skills needed by employers and final year 
students from Higher Education Institutions in East Africa revealed a significant 
difference in prioritization of the employability skills between employers and final 
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year students (Kalufya & Mwakajinga, 2016). Another study assessing perceptions 
of academics, employers, and civil servants regarding graduates’ employability 
skills in East Africa revealed general perceptions that graduates’ skills are insuffi- 
ciently developed (Guardia et al., 2021). An Inter University Council for East Africa 
(IUCEA) survey polling employers across the region concluded that graduates 
lacked employability skills including technical mastery and basic work-related 
capabilities (IUCEA, 2014). Additionally, a study was conducted in Kenya in 2017 
to explore ‘soft skills’ nurturing among Technical and Vocational Education and 
Training institutions and preparation of students for self-employment. The study 
showed that Technical and Vocational Education and Training institutions did not 
nurture soft skills such as time management, problem solving, decision making, and 
creativity and others for their survival and self-employment (Murgor, 2017). In 
Uganda, the Skilling Uganda Business, Technical and Vocational Education and 
Training Strategic Plan affirms that more than 60% of large and medium sized com- 
panies consider the training provided by such institutions to be irrelevant to their 
requirements (Mitana et al., 2019). According to the IUCEA, over half the graduates 
from the East Africa Community region lack the necessary skills to be employed in 
the labour market and over 40% of them lack adequate soft skills or life skills 
(IUCEA, 2014). While those going to school and graduating from higher education 
institutions lack adequate skills for employment, even more concerning is the large 
number of youth in East Africa who are Not in Education, Employment or Training. 
These youth, along with the more than 20% of East Africa youth who are recorded 
as unemployed, make up a large segment of the youth population. This points to a 
dire need to address the shortcomings in the education sector that are signified by 
high dropout rates and exit from schooling without adequate skills and qualifica- 
tions to progress in life. 


2.2 Learning Assessments as Evidence for Influencing Policy 


Learning assessment data have been used by education systems and development 
partners to shape education policy and practice. Notably, UNESCO’s Institute of 
Statistics (UIS) has identified several cross-national learning assessments that meet 
the criteria to measure the proportion of children and young people achieving mini- 
mum proficiency in reading and mathematics to inform monitoring under the 
Sustainable Development Goal 4. International large-scale assessments drawn upon 
include the Programme for International Student Assessment (PISA), Trends in 
International Mathematics and Science Study (TIMMS), and Literacy and Numeracy 
Assessment (LaNA). 

Regionally, most notable large-scale assessments are Southern and Eastern 
Africa Consortium for Monitoring Education Quality (SACMEQ), Programme for 
the Analysis of Education Systems (PASEC), Pacific Islands Literacy and Numeracy 
Assessment (PILNA), Southeast Asia Primary Learning Metrics (SEA-PLM), and 
Regional Comparative and Explanatory Study (ERCE). Other assessments that are 
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implemented both regionally and nationally are Early Grade Mathematics 
Assessment (EGMA) and Early Grade Reading Assessment (EGRA). UNESCO, 
UNICEF, and the World Bank as development partners, also conducted the 
Measuring Early Learning Quality and Outcomes (MELQO) program. 

Regional assessments are of course facilitated at country level. For example, 
UNICEF administered MELQO in collaboration with the National Examination 
Council of Tanzania in 2015. In Kenya, the Kenya National Examination Council 
administered SACMEQ. True national assessments are typically undertaken by 
ministries of education. For example, in Uganda, the Uganda National Examination 
Board administers the National Assessment of Progress in Education (NAPE) 
focused on Grades 3 and 6; since 1996 at primary school level and 2008 at second- 
ary school level. NAPE collects data on cognitive and non-cognitive skills, and 
findings are reported at national and sub national levels. 

Learning assessments such as these have contributed to informing the formula- 
tion, implementation, and review of educational policies and practices globally. For 
instance, Lockheed et al. (2015) established that donors use PISA results in dia- 
logue with middle-income countries to set agenda relating to reform of curricula, 
standards, and teacher professional development. 

In East Africa, the Uwezo citizen-led assessment has been implemented for more 
than 6 years across Kenya, Tanzania, and Uganda, and has focused on assessing 
learning outcomes related to numeracy and literacy. Its aim is to provide credible 
data that stimulates policy dialogues and drives educational reforms by using the 
clarion call ‘Are our children learning ?’. Although with varying degrees of accep- 
tance and acknowledgment by the respective governments, the Uwezo assessments 
in the three countries have helped to draw attention and spark dialogue on the poor 
learning outcomes across schools in the three countries (Nakabugo, 2021). 


2.2.1 The Pitfalls and Challenges of Evidence Use 


Education systems worldwide face pressure due to the increasing number of chil- 
dren entering formal education, and to the need to deliver education of high quality 
that is relevant for the fast-changing and complex world. Research-based evidence 
can positively impact quality, design, and effectiveness of policies (Goldman & 
Pabari, 2021). In fact, better utilization of research and evidence in development 
policy and practice can help save lives, reduce poverty, and improve the quality of 
life (Young, 2005). Using research to inform policy and practice: (1) ensures that 
policy decisions are informed by and anchored on a solid evidence base; (2) mini- 
mizes policy failures and waste of resources on unworkable programs; and (3) can 
drive innovations which have the potential to transform societies. One example of 
evidence generation both informing and resulting from national level policy and 
practice that contributes to global transformative development is the monitoring and 
gathering of data for tracking the Sustainable Development Goals through UIS. The 
UIS has set up an infrastructure that facilitates sharing of data and evidence from 
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each participating country. The UIS guidelines provide for different approaches to 
be used to generate evidence as long as the data is credible, reliable, and can provide 
insights into areas or sectors being analysed. 

There is evidence that the relationship between evidence production and its uti- 
lization may not be linear (Manning et al., 2020), meaning that data availability is 
not the main reason for failure to use evidence for decision making. Studies have 
shown that even though more learning data have become available in sub-Saharan 
Africa (SSA), they have not been effectively used in educational planning. For 
instance, a three-country study on use of learning assessments across Kenya, 
Tanzania, and Uganda revealed several factors that hinder the uptake and utilization 
of education data and evidence in education policy making (GESCI, 2022). The 
main factors included poor conceptualisation of learning assessments, minimal 
capacity to interpret data and evidence, poor attitudes towards learning assessments, 
weak institutionalization of research, weak dissemination, and poor stakeholder 
engagement (GESCI, 2022). Another study that covered Ghana, Guinea, Namibia, 
Senegal, The Gambia, and Zambia found that lack of capacity to analyse learning 
data, lack of assessment data integration with other data sources, missing feedback 
loops between different administrative levels, and the absence of a comprehensive 
policy framework for producing learning data, also impedes their effective use 
(IIEP-UNESCO, 2022). 

Policy making is an intricate and multi-stage process (Anderson et al., 2022). 
Influencing policy and practice is an iterative process that considers several factors, 
particularly the political and socio-economic environment where the action is tak- 
ing place. Policy influencing refers to how “external actors are able to interact with 
the policy process and affect the policy positions, approaches and behaviors” (Court 
et al., 2006, p. 6). In reality, policy making does not follow a linear approach and 
usually involves both formal and informal processes across a range of actors and 
institutions until the intended policy influencing objective is reached (Court et al., 
2006). According to a study on experiences with evidence-informed policy, influ- 
encing is often a complex web of processes that requires different ingredients 
including: a robust evidence base, building coalitions, learning the rules of the game 
in many different systems, and a process of continuous reflection and change in the 
light of experience and context (Mayne et al., 2018). 


2.2.2 Addressing Gaps in Evidence and Policy 


The Assessment of Life Skills and Values in East Africa (ALIVE) intentions were 
stimulated both by the documented gap between labour market and appropriately 
skilled workforce and by the challenges experienced in the education systems in 
East African countries, specifically Kenya, Tanzania, and Uganda as they responded 
and adapted to the growing market demands, through reforms in the formal educa- 
tion system. These two categories of concern overlap in several ways, with their 
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foundations rooted in societal norms about education. Three specific responses 
from ALiVE are discussed in this section. 


2.2.2.1 Evidence and Capacity Limitations 


The first gap addressed by ALIVE is the absence of a large-scale assessment of life 
skills and values, and the limited capacities of practitioners in the region to generate 
this evidence. Although there is evidence of large-scale assessments impacting pol- 
icy, as noted above these have primarily targeted literacy and numeracy achieve- 
ment. In the life skills and values space, there have been few large-scale assessments, 
although there are some tools that target child wellbeing and social-emotional 
development. The tools are mainly from the United States and other high-income 
countries. Some of the most popular include the International Development and 
Early Learning Assessment (IDELA; Save the Children, 2019), MELQO (UNESCO 
et al., 2017), the Social-Emotional Assessment/Evaluation Measure (SEAM; 
Squires et al., 2013), the Child Behavior Questionnaire (CBQ; Goodman & Scott, 
1999), and International Social and Emotional Learning Assessment (ISELA; D’Sa 
& Krupar, 2019). Their focus is mainly on the early years and not the pre-pubescent 
or adolescent years. In terms of types of assessments, many are self-rating scales, 
and none of them have been developed for SSA contexts. A study commissioned by 
Echidna Giving in 2018 (Wamahiu & Bapna, 2019) that mapped life skills imple- 
mentation across East Africa revealed that although organizations in East Africa are 
working on nurturing life skills and values, few are using contextually relevant 
tools. In most organizations that are engaged in the programs, test and scale devel- 
opment expertise, and large-scale assessment capacity, is low. ALiVE’s response 
has been to design, plan and implement a large-scale assessment of life skills and 
values across Kenya, Tanzania, and Uganda. 


2.2.2.2 Limited Contextual Understanding of Life Skills and Values 


The second gap addressed by ALIVE is the absence of contextual definition of life 
skills and values in East Africa. Terms used to describe generic cognitive and social- 
emotional skills or life skills vary in the education space. Values, and how they are 
defined or understood, are intertwined in a cultural, social or political context. 
Wamahiu and Bapna (2019) revealed that in East Africa there is no common under- 
standing of these skills, and little understanding of how to develop or measure them. 
This situation has also been documented in Asia and the Asia-Pacific (e.g., Care & 
Luo, 2016). Education systems are faced with the need to define and describe these 
skills in ways that make sense to the broader population and especially to parents, 
in ways that will engage community support to change perceptions of what educa- 
tion provides and what this might mean for how different qualities in children might 
be valued. Without these common understandings, the historical prioritization by 
parents and teachers of academic scores at the expense of other competencies 
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(Kenya Institute of Curriculum Development, 2016), will continue the practice of 
underfunding life skills programs and education (Fagan & Mihalic, 2003). 


2.2.2.3 Limited Policy Uptake of Evidence by Government 


The notable disconnect between written intentions and learning outcomes, particu- 
larly in life skills and values, is a third gap addressed by the ALIVE initiative. The 
growing demand for the development of generic competencies across the globe has 
prompted over 100 governments in the past couple of decades to introduce these 
complex skills into their policies and curricula (Care & Kim, 2018). This has been 
the case for governments in East Africa, with the Tanzanian government being the 
first to develop a competency based curriculum in 2008 and more recently Kenya 
and Uganda doing the same. Despite progress in integrating values and life skills in 
curricular frameworks, actual implementation varies from country to country. For 
example, in Kenya, life skills and values are in the curriculum, but according to 
some sources there is little assessment of the same, and teacher’s understandings of 
these competencies are low (Atikiya, 2021). There is progress however with the 
Kenya Institute of Curriculum Development and the Kenya National Examinations 
Council in 2023 providing guidance on designs for assessment and developing 
assessment frameworks. The recent pattern in Uganda is similar; the new secondary 
curriculum includes ‘generic skills’, and the Ugandan education system is now 
faced with operationalising the acquisition of these ‘essential skills’ (Mitana et al., 
2019). In Tanzania, life skills have been integrated into various subjects at the sec- 
ondary level with reporting in 2018 that at that time teacher’s capacities and assess- 
ment practices were lacking (UNICEF, 2018). There is a clear pattern of policies 
and curricula on non-cognitive competencies and life skills being hindered by 
examination-oriented classroom practice (Allen et al., 2016). Notwithstanding these 
challenges, in all three countries there is an acknowledgement that co-created and 
contextualised tools are needed to assess outcomes in these areas (Wamahiu & 
Bapna, 2019). 


2.3 The ALIVE Way 


ALiVE was designed to generate evidence to influence policy and practice in the 
incorporation and assessment of contextualised life skills in the education sectors of 
Kenya, Tanzania, and Uganda. ALiVE is a collaborative initiative of RELI’s Values 
and Life Skills (VaLi) cluster, composed of nearly 20 civil society organizations 
promoting values and life skills across the region. Participating organizations were 
experiencing similar challenges related to values and life skills education across the 
three countries, and so worked towards finding answers to three critical questions: 
(1) How are life skills and values understood in context? (2) How are life skills and 
values assessed in context? (3) What works in nurturing life skills and values? 
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ALiVE’s findings in response to the first two questions were accompanied by the 
expectation that policy uptake of evidence would answer the third question. 

The initiative was conceptualised as three pillars which also define the antici- 
pated outcomes: 

an assessment pillar designed to ensure that evidence on contextualised life skills and val- 

ues was generated; an evidence-led advocacy pillar designed to ensure that evidence gener- 


ated would inform policy and the public; a learning community pillar designed to strengthen 
local capacities. 


ALiVE’s theory of change (Fig. 2.1) illustrates how the three pillars lead to the 
ultimate outcome of learners being equipped with values and life skills. Each of 
these pillars overlap and result in key outputs contributing to impact. As contextual- 
ised assessments are co-created by local experts, local capacities evolve. As these 
local experts interact within a broader learning community of civil society and gov- 
ernment, East African voices become amplified regionally and internationally. 


Education 
systems focus 
on Vali / Learners 
equipped with 
values and life 
skills 


collaborative Action 


EA's 
Ti Learning 


amplified Community 


Global 


Fig. 2.1 ALiVE theory of change 
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2.3.1 Pillar 1: Contextualised Assessment 


The pillar was designed to respond to the low capacity in the region on how to 
develop life skills and values assessments and to the lack of evidence on life skills 
and values in East Africa. The pillar focused on capacity enhancement of local 
experts and practitioners, particularly curriculum and assessment experts from the 
participating governments, as well as Regional Education Learning Initiative 
(RELI) members, to develop expertise in the assessment of life skills and values. 
Through this capacity enhancement process, evidence on adolescents’ life skills and 
values could be used for awareness raising and policy advocacy. The pillar consisted 
of creating an assessment tool based on contextualised definitions of the identified 
competencies. The competencies were selected through a consultative process that 
included scanning of the curricula of Kenya, Tanzania, and Uganda and a consensus 
process with the VaLi members of RELI. The selected competencies were three life 
skills (collaboration, problem solving, and self-awareness) and the value of respect. 
The tool development process was undertaken through five workshops over a period 
of 10 months followed by a pilot phase. The pillar achieved the conduct of a 
household-based assessment of the ALIVE survey tool to over 45,000 adolescents 
across Kenya, Tanzania, and Uganda, hence producing large-scale evidence of func- 
tioning levels of adolescents on the targeted competencies. The process adopted 
three features to make it a viable mechanism for generating evidence to influ- 
ence policy. 


e Contextualisation: Contextualisation was a process to identify local understand- 
ings and definition of the skills. An ethnographic study generated an understand- 
ing of how the selected ALIVE competencies were defined in local environments. 
Over 350 participants in villages across 15 districts in Kenya, Tanzania, and 
Uganda, were interviewed to collect a first round of information. The districts 
represented diverse groups of people including fishing, pastoralist, agricultural, 
semi-urban, urban slums, and rural communities. Language diversity was catered 
for with over 17 different languages represented. Within these communities, 
leaders, parents, teachers, social workers, and others working with children, 
along with adolescents, were targeted and purposively selected to be part of the 
study. Key information interviews generated an understanding of how these 
groups define the chosen competencies. Participants were asked questions about 
how they defined the competencies, the terminologies for these in their language, 
the characteristics of those who have these competencies, and so on. 

e Tool Development: The tool development process involved local practitioners 
and teachers who used their understanding and experience of and with adoles- 
cents in their context. The development team included technical experts from 
curriculum and examination councils, representatives from civil society, local 
colleges and universities including those with experience in psychometrics and 
psychology. The team for the ALiVE tool development process comprised 47 
individuals who worked virtually and in-person over the period of creation, pilot- 
ing, and finalisation. The participants were taken through a process of defining 
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the skills based on the contextualised study, developing skills frameworks based 
on the skill structures, developing tasks based on the context, pre-testing the 
tasks to check for validity, and piloting the tasks. The tasks designed were either 
scenario or performance-based tasks. This was a major point of departure from 
the self-rating scales predominantly used to measure life skills and values. 

e Household-based Assessment: The household-based mode of assessment was a 
key characteristic of ALIVE, reflecting the emphasis on all adolescents, rather 
than only those attending formal education. It also reflected the focus on context 
and environment by ensuring appropriate sampling of the target population 
across the three countries. Household assessments generate rich data on target 
competencies but also capture socio-economic and environment information. 
This contextual information provides insights to strengthen evidence generated. 
For example, competencies can be analysed against schooling, gender, geogra- 
phy, socioeconomic status, ability, and age. The assessment process started with 
a training of trainers where the development team built the capacity of trainers in 
each country. These trainers then trained over 3000 test administrators across the 
three countries. For the assessment itself, the test administrators worked in pairs 
going from household to household over a three-day period in each enumeration 
area. Ultimately 45,442 adolescents were assessed across the three countries. In 
each village or enumeration area, test administrators were escorted to each 
household by community leaders or village chiefs. 


2.3.2 Pillar 2: Evidence-Led Advocacy 


This pillar was designed to represent the evidence generated from the assessment 
pillar to policy makers, practitioners, parents and the community about the status of 
life skills and values in the three countries. The pillar drew on identification of key 
messages from the large-scale assessment results and was nuanced through identifi- 
cation of key stakeholders at local, national, and regional levels. Targeted outreach 
to media was a core strategy to raise visibility, particularly associated with events 
including government representatives; while establishing technical credibility was 
attended to through publication and presentation in professional media. Additional 
information about the philosophy and approach undertaken by ALiVE can be found 
in Mugo (2024; Chap.12, this volume). 

The key messages were determined on the basis of analysis of the adolescent 
response data. High-level findings were packaged into reports for dissemination. A 
report was developed for each of the four education systems (Kenya, Tanzania main- 
land, Uganda, and Zanzibar) with key findings for each skill. Analysis of the results 
showed how adolescents performed across gender, age, schooling, and ability cate- 
gories. Also reported were district comparisons, associations between the compe- 
tencies, and associations between these and basic literacy and digital literacy skills. 
A regional summary report was also launched, introducing an approach to reporting 
results within a learning framework as distinct from a summative achievement 
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approach. The learning framework approach is an intentional strategy for ALIVE in 
emphasising integration of life skills and values within overall education provision. 

Identification of key stakeholders was undertaken through liaising with members 
of RELI and central government units within the four education systems. The dis- 
semination process was launched at national level in each country, then moved to 
the district and community levels, simultaneously including engagement with media 
channels. Targeted outreach was designed to capture media interest in general edu- 
cation reporting, and the visibility of ministers and high officials in the education 
systems. 

Concurrently with direct advocacy to government, ALIVE represented its learn- 
ings through preparation of technical reports to establish credibility in the assess- 
ment community, publications in academic literature, and presentations in local, 
regional, and international events. 


2.3.3 Pillar 3: Learning Community 


ALIVE took a self-conscious and deliberate approach to learning. The intention was 
to strengthen local capacity in assessment and representation of evidence to effect 
change. At a RELI organisational level, the intention was to create a unified regional 
voice as to how life skills and values could be nurtured and assessed. The approach 
involved three elements: learning by doing; learning from and with one another; and 
learning for learning. Through the learning by doing approach, local experts were 
engaged and able to benefit from the development work including conducting litera- 
ture searches and reviewing academic referencing tools; development and refine- 
ment of constructs; tool development and assessment; data analysis and reporting; 
and academic authoring and publication. Learning from and with one another 
involved the establishment of a learning community that were engaged through 13 
‘learn-shops’ organised over a 3-year period. Over 2000 participants were drawn 
from the RELI network, government, academia, and collaborating institutions. 
Global actors in the life skills space such as Amplify Girls, EASEL-Lab at Harvard, 
OECD, INEE, The Life Skills Collaborative, among others, facilitated and contrib- 
uted to the learning. A culmination of this learning from and with one another was 
the VaLi-Africa conference, the first of its kind in East Africa, bringing together 
national and global players in the life skills and values space. As a result of these 
learning processes, individuals have been empowered to amplify their voices in the 
space of life skills and values through presentations at conferences and meetings 
reaching over 10,000 participants. In the learning for learning approach, the ALiVE 
team underwent a process of documenting the learnings of ALIVE. This process 
yielded five learnings briefs as well as a formative learning framework which are 
now being used to inform a second phase of the ALIVE initiative. Additional infor- 
mation about the learning approach undertaken by ALiVE can be found in Turner 
et al. (2024; Chap.11, this volume). 
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2.4 Conclusions 


ALIVE has taken an intentional step to gather evidence to inform growing concerns 
about the nature of education in the participating countries. The use of that evidence 
is the most critical element in ALiVE’s agenda, but is totally reliant on the genera- 
tion of high quality data which is valid for purpose. ALIVE has contributed to global 
expertise through its generation of large-scale, household-based, multi-national 
assessment of ‘non-traditional’ competencies, namely life skills and values. While 
the utility of large-scale assessments is sometimes hindered by factors such as lim- 
ited capacity of policy makers to engage with evidence, poor dissemination strate- 
gies, and inadequate engagement of key stakeholders, ALiVE adopted strategies to 
tackle these barriers throughout its development, implementation, reporting, and 
dissemination processes. And while local stakeholders’ voices are sometimes lim- 
ited in reach, ALiVE’s focus on learning and building a learning community of both 
local and global actors, is strengthening local expertise and amplifying local voices 
to the global stage. 
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Abstract The world is concerned about young people’s preparedness to face chal- 
lenges in the workplace, as well as society’s ability to respond to the social and 
economic issues of the twenty-first century. To respond to this challenge in the past 
decade, the education systems in East Africa have incorporated life skills and values 
into their policies and curricula; however, the actual implementation and incorpora- 
tion of teaching and learning practices that foster these skills in the classroom is 
mostly unexplored. It has also been noted that tools used to measure 21st century 
skills in non-Western contexts have been borrowed from Western literature. This 
leaves no room for different understandings and conceptualisations of the skills to 
be measured. The Assessment of Life Skills and Values in East Africa (ALIVE) 
team addressed the gap in existing literature by exploring the understanding of col- 
laboration, problem solving, self-awareness, and respect in the East African context 
through rapid ethnographic interviews. Each of these constructs are represented in 
the education systems of Kenya, Tanzania, and Uganda. The researchers interviewed 
a total of 368 participants (80 from Kenya, 55 from Tanzania, and 95 from Uganda). 
Of these, 76 participants were adolescents; 78 were parents; and 76 were teachers. 
What emerges prominently in the East African context is that personal identity 
incorporates more communitarian than individualistic features compared to the 
Western descriptions of the self. As a consequence, when designing a data collec- 
tion tool for assessing life skills in the East African context, there are several con- 
ceptual, ethnographic, and epistemological elements to be considered—not only at 
the initial stage of conceptualising the framework of a tool, but also in the process 
of tool development, data collection, and data analysis. 
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3.1 Introduction 


3.1.1 The Importance of Twenty-First Century Competencies 
and Their Relevance to the Global and East 
African Contexts 


The dawn of the twenty-first century has brought rapid and sweeping technological 
and cultural changes to the entire world. As a result, we are facing more numerous 
and complex problems that require a radically different set of competencies and 
way of thinking. In a constantly changing world, it is difficult to anticipate the 
knowledge or skillset necessary for professional success, since it is increasingly 
challenging to predict the types of jobs we will be doing. We do know that there is 
an ever-growing need for individuals who can process data, evaluate concepts, and 
develop arguments. This kind of skillset belongs to individuals who possess the high 
levels of social understanding and critical-thinking skills needed to make intelligent 
judgements about public issues (Fong et al., 2017; Pascarella & Terenzini, 2009). 

Employers are concerned about young people’s preparedness to face challenges 
in the workplace, as well as society’s ability to respond to the social and economic 
issues of the twenty-first century. By 2030, the youth workforce will be over 40% of 
the population in developing countries, with approximately 11 million young peo- 
ple entering the labour market each year (World Bank, 2015). This could potentially 
stimulate the economic development of these countries and reduce poverty, but 
young people must acquire the necessary skills to access and retain productive jobs. 
While employers value academic skills in young people, they are increasingly pri- 
oritising life skills and good values (Rosekrans & Hwang, 2021). 

The complexity of contemporary life reveals the inadequate nature of traditional 
learning methods; education systems must adapt if they are to equip youth with the 
competencies that the market demands. In this regard, the Learning Metrics Task 
Force convened by the Brookings Institution and UNESCO’s Institute of Statistics 
coordinated a global consultation to improve students’ learning outcomes in 118 
countries, including 11 countries in Africa. The task force highlighted that in addi- 
tion to numeracy, literacy, science, and technology, learning should include stu- 
dents’ physical, social, and emotional wellbeing, as well as actual learning to learn 
(Anderson & Ditmore, 2016). The World Economic Forum (Boston Consulting 
Group, 2015) conducted a meta-analysis of research in primary and secondary edu- 
cation and clustered the skills needed in the twenty-first century into three macro 
categories. First, foundational literacies include the skills that students apply to 
everyday tasks such as literacy, numeracy, ICT literacy, and basic knowledge of the 
cultural environment in which they live. The second, competencies, describes the 
qualities of students who know how to deal with complex challenges. Critical think- 
ing, problem solving, creativity, collaboration, and communication are the most 
relevant competencies. Finally, character qualities, such as persistence, grit, adapt- 
ability, leadership, and curiosity describe how students approach a challenging 
environment. 
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In the East African context, recent labour market surveys (Guardia et al., 2021; 
Omala et al., 2016) highlight the demand for a workforce that possesses ‘soft skills’ 
and higher-order thinking skills. The 2019 Echidna Giving Life Skills and Mindset 
Change Project conducted a study that recognised that for several decades, minis- 
tries of education and civil society organizations in Kenya, Tanzania, and Uganda 
acknowledged the effectiveness of life skills, especially in combating HIV/AIDS 
and sexual and gender-based violence, as well as resolving conflicts, promoting 
peace, nurturing leaders, empowering girls, advancing gender equality, and chang- 
ing mindsets. There is, however, limited evidence on how this happens, and there is 
no consensus on the correct approach to fostering these skills from the child’s expe- 
rience in family and community and through the education system (Deitz et al., 
2021; Rosekrans & Hwang, 2021). 


3.1.1.1 The Relevance of Twenty-First Century Competencies for Revised 
School Curricula in Kenya, Tanzania, and Uganda 


Sustainable Development Goal 4.7 (SDG) emphasises the importance of ensuring 
that all learners acquire not only knowledge but also skills to promote sustainable 
lifestyles and a culture of equity and excellence. Accordingly, quality education 
should aim to foster not only cognitive learning objectives but also social and emo- 
tional objectives (Care et al., 2018a, b). 

Despite this recognition and the explicit demands of the labour market, the edu- 
cation systems have yet to make widespread changes (Jukes et al., 2018). A study 
(Care et al., 2018b) of policies, curricula, and vision statements from over 150 gov- 
ernments around the world highlighted a gradual increase of a range of skills and 
competencies—such as communication, critical thinking, problem solving, and col- 
laboration—in policies and curriculum statements. However, the study also showed 
that almost 90% of the countries has yet to take the necessary steps to implement 
those polices and bring about a transformational shift in the pedagogical approaches 
used in the classrooms (Care et al., 2016, September; Care & Anderson, 2016). The 
challenges are political as well as procedural and technical. In the East African con- 
text, the focus on competencies requires a shift in the learning theory, including 
pedagogical and assessment practices (Cheptoo & Ramadas, 2019). 

In Kenya, Tanzania, and Uganda, students’ learning has remained inadequate, 
despite the fact that Free Primary Education has increased enrolment. The education 
systems in these countries are not effectively helping children attain life skills and 
good values (Cheptoo & Ramadas, 2019). To respond to this challenge in the past 
decade, the education systems in East Africa have incorporated life skills and values 
into their policies and curricula; however, the actual implementation and incorporation 
of teaching and learning practices that foster these skills in the classroom is mostly 
unexplored. At the time of writing, for instance, teacher education curricula and con- 
tinuous professional development policies do not nurture teachers’ competencies in 
enhancing life skills and values, due to lack of clarity on the pedagogical practices that 
can enhance such skills and values in schools. There is also little evidence on whether 
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teachers themselves possess the life skills and values they need to transfer to learners. 
Moreover, teachers lack knowledge of what works best to enhance and assess these 
competencies in the classroom (Giacomazzi et al., 2022a). The following paragraphs 
describe how the governments of Kenya, Tanzania, and Uganda have incorporated 
education for these competencies into their education systems. 

The stated goal of Kenya’s education philosophy is to prepare students for social 
cohesion, human growth, and economic development. The need to make Kenya’s 
education philosophy more explicit (within policy documents) can be traced back to 
the Koech Report and subsequent Sessional Paper No. 1 of 2005 on education 
(Ministry of Education Science and Technology, 2005), research, and training. 
Before the Koech Report, most education reforms were explicit in terms of educa- 
tional goals, but the philosophy was somewhat implicit. The current Kenya 
Competency Based Curriculum (CBC) philosophy aims to provide holistic, quality, 
and inclusive education and training for greater social cohesion and sustainable 
development. It devotes special attention to the role of life skills and values at all 
stages of education, from preschool to senior/upper secondary school (Heto et al., 
2020). The expectation is that a student can acquire specific values that encompass 
the moral, socio-cultural, civic, environmental, cognitive, and physical dimensions 
at each level. It therefore adopts a more holistic approach to a student’s formation. 

In Tanzania, the history of CBC can be traced back to a philosophy of education 
based on self-reliance, which was promoted by Nyerere (1967). This curriculum 
aimed at preparing learners to cope with life’s problems in creative and innovative 
ways. The CBC was developed and adopted in 2005 as the result of a training sys- 
tem gap that negatively affected the quality of graduates being sent into the job 
market. The previous curriculum was heavily content-based and was not explicit 
about the kinds of competencies needed. The purpose of the CBC in Tanzania is to 
equip students with creative and inquisitive minds as well as employability skills 
needed to perform different roles. Challenges nevertheless persist because students 
have not been able to practice and apply what they have been learning such that they 
can solve societal problems as was envisaged (Nkya et al., 2021). 

For many years Uganda’s philosophy of education has been criticised for not 
responding to the needs of society, because the country’s mode of teaching is largely 
content-centred and exam-oriented. This traditional mode of education has been 
blamed for producing individuals with no values or skills. To address such public 
concerns, the Ministry of Education and Sports through the National Curriculum 
Development Centre (NCDC), set out to review the curriculum at all levels. The 
review process began with the development of an Early Childhood Development 
Curriculum (ECDC) framework in 2005. This was followed by the review of the 
primary curriculum, which emphasised the use of a familiar language at the early 
stages of learning. The biggest highlight of the review, however, was the launch of 
the new lower secondary school’s competency-based curriculum (NCDC, 2019). 
The new curriculum emphasises what learners are supposed to do, not what they are 
supposed to know. This makes it adaptable to society’s evolving needs and presents 
the country with an opportunity to nurture learners who possess what the curricu- 
lum refers to as ‘generic skills’ and values. 
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3.1.1.2 Reconciling Western Conceptualizations of Twenty-First Century 
Competencies with East African Perspectives 


While the development of twenty-first century skills is perceived as key to a young 
person’s achievement of positive academic and life outcomes around the world, it is 
evident that most of the research studies on this topic have been dedicated to the 
investigation of these competencies in Western, industrialized, and high income 
contexts (Giacomazzi et al., 2022a; Jukes et al., 2021a). This is concerning if we 
consider the case of sub-Saharan Africa. Even though it is one of the youngest and 
most populated regions in the world, there are few studies dedicated to the investi- 
gation of life skills and values in this context (Deitz et al., 2021; Jukes et al., 202 1b; 
Rosekrans & Hwang, 2021). It has been noted that tools used to measure these 
competencies in non-Western contexts have been borrowed from Western literature 
(Jukes et al., 2021a). This leaves no room for different understandings and concep- 
tualizations of the skills to be measured. Moreover, while interventions aimed at 
enhancing social and emotional skills in Western contexts consistently demonstrate 
positive academic and behavioural results (Taylor et al., 2017), in non-Western con- 
texts, evidence is less conclusive (Deitz et al., 2021; Rosekrans & Hwang, 2021). 
This might suggest that the measurement tools or the interventions implemented do 
not respond to the needs in all contexts. Moreover, there are studies that demon- 
strate differential impact of the same intervention in different contexts (Deitz 
et al., 2021). 

According to the science of cognition, children’s development is influenced by 
the culture and language of their community. Culture, as a system of shared beliefs, 
customs, and values of a community or society, shapes how people make sense of 
their reality and determines how they interact with it. These factors play a key role 
in defining which life skills and values a specific group of people deems worthy or 
desirable (Giacomazzi, 2022; Jones et al., 2021). Through interaction with their 
community of origin, children are exposed to cultural practices that contribute to 
their development (Bronfenbrenner, 1986). According to Rogoff (2003), a child 
grows while participating in the community; it is a process of transformation 
through participation in the group of people to which one belongs. It is for this rea- 
son that life skills and values can be understood only through the lens of culture; 
they may vary and evolve differently, depending on cultural circumstances. 

As shown, governments in East Africa are clear about the need for children to 
develop twenty-first century skills, under umbrella terms such as ‘core competen- 
cies’ and ‘generic skills’. Kenya, Tanzania, and Uganda have all rolled out 
competency-based curricula which include these skills, but the three countries are 
now grappling with the challenges of implementing these key reforms. Across the 
world, the implementation of such reforms requires a shift in learning and assess- 
ment practices, which remains challenging for education systems (Anderson & 
Ditmore, 2016). 

The first challenge that needs to be addressed is the lack of understanding of the 
nature of these skills and how they are manifested and perceived by the local contexts. 
A recent systematic review of the literature on the topic (Giacomazzi et al., 2022a) 
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highlighted that few researchers who aim to assess or enhance life skills in sub- 
Saharan countries have recognized the importance of starting from the local under- 
standings of the skills, and few studies have investigated these local understandings 
(Jukes et al., 2021b). The lack of attention to the local interpretations of a skill 
results in the development of pedagogical approaches and assessment tools that are 
irrelevant to learners (Giacomazzi et al., 2022a). This has clear repercussions for 
what happens in the classroom, since the way the curriculum presents the skills 
shapes the learning objectives and the pedagogical and assessment strategies in the 
classroom (Care et al., 2018b; Jukes et al., 2021b). It is therefore paramount to 
investigate the skills constructs that the curricula aim to develop, and so identify and 
describe the elements that might demonstrate that a person possess the skills in a 
specific context. 

Another important element in understanding life skills is the fact that the demon- 
stration of skills differs in relation to a person’s stage of development and level of 
proficiency in that skill. The development of skills implies the ability to apply the 
skills to increasingly complex situations. The skills are gradually and progressively 
acquired in more sophisticated ways, which can be described differently depending 
on the context and targeted population (Care et al., 2018b). Nurturing a skill implies 
the need to identify a learning progression that applies across contexts in which the 
skill is demonstrated and can be enhanced. The concept of learning progression is 
common in most educational contexts and is widely understood by teachers and 
students. For example, in learning mathematical concepts, one builds on sequen- 
tially defined steps, and at each level, teachers can identify what they can expect 
from their students. Similarly, if teachers aim to enhance a specific skill in students, 
they should describe how students develop the skill at different levels of complexity 
(Care et al., 2018b). Yet with twenty-first century skills, the evidence of what works 
at different levels of sophistication is not very clear. 

Another challenge is related to how one should measure twenty-first century 
skills. The lack of knowledge about the learning progressions that describe these 
skills poses major challenges in terms of how to assess these skills. As a conse- 
quence, the majority of the tools used to assess these skills in sub-Saharan Africa 
present issues of reliability and validity (Deitz et al., 2021). Measurement tools that 
were validated in one context might not be relevant to another. 

To conclude, while the process of globalization is unavoidable and heavily influ- 
ences the conceptualization of these generic skills, we should look to promote inter- 
cultural competency (Deardorff, 2006), a plurality of epistemological beliefs and 
worldviews, and sensitivity to specific contexts (Assié-Lumumba, 2016; Lee, 2017). 
Constructs developed within Western culture may not be universally applied to all 
contexts; their transferability to the local context must be questioned and investi- 
gated (Lee, 2017). In the East African context, the system reforms and the practices 
of nurturing and measuring generic skills should promote the use of methods and 
tools that are respectful of the local perspectives. The starting point for development 
and use of such tools should be the contextual definition of these skills. 
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3.2 Method 


In August 2020, the Assessment of Life Skills and Values in East Africa (ALiVE) 
team addressed the gap in existing literature by exploring the understanding of col- 
laboration, problem solving, self-awareness, and respect in the East African context. 
Each of these constructs are represented in the education systems of Kenya, 
Tanzania, and Uganda. 


3.2.1 Study Design 


A qualitative approach allowed deep immersion into local realities through rapid 
ethnographic interviews: an approach previously used in Uganda (Boothby et al., 
2017). In a rapid ethnographic study, the timeframe for fieldwork is limited; such 
studies are usually conducted in places where time and resources are not always 
available. Rapid ethnographers go into the field with a well-structured interview 
guide and a narrow set of research questions. 

The central aim of this design was to provide rich insights into how East Africa 
views life skills and values in order to document how the selected skills and values 
relate to the culture and to identify the perspectives and practices of adolescents, 
parents, and educators in their context. 

The study was conducted during COVID-19 pandemic. During the time of the 
data collection, schools in Kenya and Uganda were still closed and the communities 
were still affected by restrictions in terms of mobility. It was also a time in which 
media were often referring to the lack of like skills and values in youth as a chal- 
lenge the society had to face in times of pandemic. 


3.2.2 Sample 


The study sample included adolescent boys and girls from 13 to 17 years of age; 
parents and guardians; and key persons such as teachers, social workers, and local 
leaders from 15 districts across the three countries. The researchers purposefully 
sampled study participants in areas with the following characteristics: (a) urban, 
low-income areas within the capital city; (b) rural, agriculture-rich, and within 
100 km from the capital city; (c) rural areas, 300—400 km from the capital city; (d) 
rural, pastoralist areas, 400-800 km from the capital city; and (e) those with dif- 
ferent characteristics from those mentioned above, including fishing communi- 
ties. The study randomly sampled two villages from each selected district. From 
each selected village, at least four interviews were conducted with adolescents 
(two of each gender, and with a mix of those in primary school, secondary school, 
vocational training, and those out of school); four interviews with parents (two of 
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interviewed adolescents and two of non-interviewed adolescents, while mixing 
fathers and mothers); and four interviews with key persons (teachers, social work- 
ers, and other people who consistently work with adolescents). The researchers 
interviewed a total of 368 participants (80 from Kenya, 55 from Tanzania, and 95 
from Uganda). Of these, 76 participants were adolescents (36 males, 40 females); 
78 were parents (35 males, 43 females); and 76 were teachers (42 males, 34 
females). 


3.2.3 Tools 


The researchers conducted rapid interviews using a structured interview guide. The 
participants were asked to reflect on a specific skill (or value) and think about pos- 
sible words or ways of defining the skill. The researchers then invited them to state 
and explain it in their mother tongue. Then they asked the participants to think 
about an adolescent (aged 13—17) they knew who clearly demonstrated the skill. 
They also invited the participants to reflect on the behaviours this adolescent dis- 
played and on how they could assess whether another adolescent had that skill. 
Finally, the interviewer would ask a question about the systems that could allow an 
adolescent to nurture the skill in his or her life (i.e., family, school, peers, etc.). 


3.2.4 Coding and Data Analysis 


The analysis of the content generated codes related to the definition of the con- 
struct as described by the participants, the set of subskills or dimensions related 
to each construct, the dispositions and values the adolescents displayed, the 
suggested methods to assess the skill, and the support systems and other factors 
that could enhance the skill. As recommended by Gibbs (2018) and using the 
Dedoose program (version 8.3.41.), the researchers conducted an analysis of the 
understanding of each skill and the one value—paying specific attention to ele- 
ments of contextualisation in contrast with what had been established from the 
literature review. 


3.3 Discussion 


The full reports of the contextualisation studies elaborate on the local conceptuali- 
sation of the three skills and the value that were investigated (RELI, 2023). This 
section discusses the key features that emerged: the communitarian understanding 
of the self; the importance of nurturing positive relationships; collaboration as 
means to help the community; and the strong focus on respect. 


3 The Contextualisation of 21st Century Skills in East Africa 39 


What emerges prominently in the East African context is that personal identity 
incorporates more communitarian than individualistic features compared to the 
Western descriptions of the self, as it has been highlighted by Grosser and Lombard 
(2008): “We have to encounter the collective we, before we encounter the collective 
I. I am only a person through others” (p. 1368). 

In problem solving, for instance, the skill is mostly aimed at resolving interper- 
sonal conflicts; the findings suggest that the life of each adolescent is closely related 
with the life of their community, whether it is the family, the school, or the larger 
society. Even in the process of problem solving, the participants highlighted how 
crucial it is for them to compare their views about challenges facing them with those 
of more experienced peers or older family or community members. In this regard, 
for instance, the Baganda tribe in Uganda promotes a sense of community by under- 
scoring the importance of respecting elders and by discouraging selfish attitudes 
that aim only at achieving personal interests (Kassimir, 1998). The findings suggest 
that self-empowerment can be achieved through fostering a sense of community 
belonging (Giacomazzi et al., 2022b). There seems to be a fine line in problem solv- 
ing between incorporating others’ advice and over-reliance on someone else’s sug- 
gestions, but this may suggest the need to rethink how epistemic beliefs and 
perspectives are usually transferred across cultures and applied to sub-Saharan 
contexts. 

The findings related to problem solving resonate with understandings that emerge 
from interviews on self-awareness in Kenya, Tanzania, and Uganda. Notably, some 
of the most frequently mentioned competencies in the contextualization study were 
self-confidence, relationship skills, communication skills, and guidance and coun- 
selling. Self-awareness is strongly influenced by the relational components of East 
African life. In this context, the values and behaviours that are more frequently 
relatable to a self-aware adolescent are related to respect, positive conduct, obedi- 
ence, discipline, humility, and fearing God. 

Working together with the aim of helping the community is a feature of the East 
African understanding of collaboration. The participants in this study did not 
emphasise common goals, but rather the self in relationship with the community. 
Adolescents in this context appear to collaborate largely in solving problems affect- 
ing their community. Here, collaboration is not reduced to the accomplishment of a 
task in the family or at school, but it is seen as a way of conceiving of oneself; col- 
laboration is driven more by the need to stick together than by the goal to be 
achieved. In the literature, collaboration is mostly established to complete a task 
that would not be possible to accomplish alone (Hesse et al., 2015). Similar to the 
findings on the self-awareness and problem solving skills, the values associated 
with collaboration are related to respect, unity, trust, and love. The value of unity 
emerges in the interviews as a synonym of collaboration (Fontana et al., 2022). 
Sharing responsibilities and having strong relationship skills are therefore consid- 
ered necessary for the process of collaboration. 

The study revealed similar patterns for adolescents who are perceived to value 
respect: they exhibit strong relationship and communication skills, guidance and 
counselling skills, teamwork or collaboration skills, self-confidence, empathy, and 
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self-regulation skills. At the same time, several participants connected respect with 
valuing others and further elaborated by referring to caring for others, considering 
others important, and recognizing their worth. On the other hand, self-respect— 
meaning one’s self-understanding and self-knowledge—did not emerge strongly 
from the interviews. 

Table 3.1 compares various frameworks of social and emotional learning skills 
with the findings of the four contextualization studies on problem solving, collabo- 
ration, self-awareness, and respect. 

The differences between the skills conceptualisations of the ethnographic studies 
and those present in the literature from Europe and North America need to be con- 
sidered when designing assessments and integrating them into the curriculum. 
When assessing life skills and values, the use of imported tools from different cul- 
tural environments may produce results that are not relevant to the situation on the 
ground and may misrepresent actual competencies. Also, lower performance of 


Table 3.1 Comparison of life skills and the value of respect across the contextualisation study and 


published literature 


Hilton & Pellegrino Clusters of twenty-first- 


Collaboration Contextualization study Century Competencies (EASEL Lab, 2023a) 
Definition Working/staying together Coordination 

Teamwork/cooperation Cooperation 

Helping the community 

Sharing 
Subskills Relationship skills Communication 


Communication 


Interpersonal skills 


Guidance & Counselling 


Empathy/perspective taking 


Goal setting 


Conflict resolution 


Self-confidence 


Negotiation 


Problem solving | Contextualization study 


IDEAL Framework (Bransford & Stein, 
1993) 


Definition Facing problems Identify the problem 
Finding solutions Define the problem 
Identifying a problem Explore possible strategies 
Judgment Anticipate outcomes and act 
Asking for advice Look and learn 
Knowing/understanding a 
problem 

Subskills Communication Attentional control 


Cooperation/collaboration 


Planning 


Guidance/counselling 


Time management 


Leadership Emotional control 
Relationship skills Organization skills 
Self-confidence 


(continued) 
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Table 3.1 (continued) 


Hilton & Pellegrino Clusters of twenty-first- 

Collaboration Contextualization study Century Competencies (EASEL Lab, 2023a) 
Self-awareness Contextualization study CASEL Framework (CASEL, 2023) 
Definition Knowing/understanding self | Accurate self-perception 

Taking care of self Identifying emotions 

Being a good person Recognizing strengths 
Subskills Self-confidence/self-esteem | Self-confidence 

Communication Self-efficacy 

Relationship skills Growth mindset 

Guidance and counselling 

Self-regulation 

Goal setting 

Planning 

Empathy 
Respect Contextualization study Sesame Workshop Global Framework for 

Learning (EASEL Lab, 2023b) 

Definition Valuing others Positive regard toward self 

Fearing god Regard for others 

Taking care of self 

Knowing or understanding 

self 

Being a good person 
Subskills Expressive communication | Self-confidence 

Relationship skills Social awareness 

Receptive communication Self-worth 

Guidance and counselling | Empathy 

Empathy Appreciate similarities and differences 

Self-regulation Show respect for tradition 

Teamwork/cooperation 

Self-confidence/self-esteem 


populations in low- and middle-income countries could be due in part to the way 
the tool was designed, to social biases, or to the lack of relevance of the tools to the 
context. 

Note that most of the tools developed in Western countries use self-report mea- 
sures (Deitz et al., 2021; Smart et al., 2019). Self-report tools offer valuable infor- 
mation about people’s perceptions of their own abilities, and the information they 
gather can predict future life outcomes. Yet people often overestimate or underesti- 
mate their performance based on their social bias. In the East African context, the 
importance people give to what the community members think about them may also 
influence respondents’ perceptions. That is why it is important that tool developers 
consider various methods of assessment, beyond self-reporting, that could reduce 
such biases. Observation protocols, performance-based or social-network measures, 
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as well as scenario-based or task-based tools might be more appropriate to assess 
life skills in a specific context. 

The contextualisation studies of the competencies investigated by ALIVE reso- 
nate with the findings of similar studies on critical thinking conducted in South 
Africa. For instance, Grosser and Lombard (2008) and Madondo (2018) advocated 
for traditional African cultures to be considered when designing critical thinking 
assessment tools. These authors showed how the African communitarian and altru- 
istic lifestyle differs from the Western individualistic one. This, for example, leads 
to the common practice in the African context of avoiding a rhetorical method of 
investigating or arguing about a problem with others. Consequently, it is quite 
uncommon to find parents and teachers who highlight the importance of argumenta- 
tion (Grosser & Lombard, 2008). For this reason, approaches aimed at assessing 
skills like problem solving or critical thinking in the classroom should consider the 
communitarian approach to life that encourages collaborative approaches to 
addressing a challenging situation (Giacomazzi et al., 2022a). Similarly, in East 
Africa, assessment methods that rely on rhetorical argumentation or dialogical 
explanation of one’s standpoint should be discouraged in favour of methods that 
allow for collaborative resolution of problems. 


3.4 Conclusions 


The contextualisation process of twenty-first century skills should begin by concep- 
tualising the skills that need to be measured in a specific setting. The ALiVE proj- 
ect’s approach to investigating the meaning of collaboration, problem solving, 
self-awareness, and respect in Kenya, Tanzania, and Uganda showed that the local 
understandings of these skills is influenced mainly by the communitarian concept of 
the self, which is a characteristic of the East African culture. 

When designing a data collection tool for assessing life skills in the East African 
context, there are several conceptual, ethnographic, and epistemological elements to 
be considered—not only at the initial stage of conceptualising the framework of a 
tool, but also in the process of data collection and data analysis. The epistemologi- 
cal beliefs of the targeted population might differ from those of the researchers; this 
conceptualisation study thus offers useful insights for the contextual definition of 
the behaviours that manifest a specific skill in the population of interest. It goes 
beyond the problem of language and of an accurate translation of the tool; the choice 
of the assessment method and of the toolbox, as well as the definition of a rubric for 
describing the observable traits of those with a low, medium, or high level of each 
component of the skill, should be driven by an in-depth understanding of the context 
in which the assessment tool is to be used. In the case of tools developed for adoles- 
cents, for example, the involvement of parents and educators who know the young 
people to be assessed, their behaviour, and their culture, is important for ensuring 
appropriateness (including age-appropriateness) and relevance. This highlights the 
importance of considering context at every level of the educational and assessment 
process. 
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Abstract Problem solving is a term that describes a vast number of processes and 
applications. Countries in East Africa, as well as globally, are looking to equip their 
young people with problem-solving competencies, which are then hoped to resolve 
the major issues that all societies confront. Accordingly, it is one of the competen- 
cies included in curricula developed by East African education systems. The devel- 
opment of conceptual and assessment frameworks by the ALiVE team was 
contextualised through reference to recent research undertaken in the ALIVE par- 
ticipating countries. Aspects of this conceptualisation differ from those used in most 
large-scale assessments; and the ALiVE approach to design and development of the 
assessment was construct-driven. This means that the form of the assessment tasks, 
coding criteria, and intended reporting style, are determined by the substance of the 
construct and how visible signs of this substance might best be captured. The frame- 
work was developed through a realistic appraisal of what would be possible to 
assess at household level. In development of the assessment tool, three factors were 
considered: the nature of the construct itself; the medium through which the assess- 
ment would be conducted; and the use to which the assessment results would be put. 
The chapter describes the formal process followed in defining problem solving and 
developing its assessment tools. A set of six workshops attended by 47 representa- 
tives of the collaborating organisations structured the process of creation; initially 
engaging at a level of simplicity to ground differing understandings of the skill, and 
then moving to the complexity of test and scale development processes. The data 
from the large-scale assessment indicate that ALiVE’s measurement of problem 
solving is robust, with the results reflecting maturation with more education. This 
means that instructional time in the context of a general curriculum is providing a 
learning environment in which problem-solving processes can be nurtured. 
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4.1 Introduction 


Problem solving is a term that describes a vast number of processes and applica- 
tions. It is used as we struggle with daily domestic logistics, used in mathematics 
classrooms as students are presented with ‘specific problems’, measured in large 
scale assessment programs to evaluate domain general problem-solving abilities, 
and referred to as wicked problems as we confront global issues. There is no doubt 
that all these applications draw on human cognition—how to analyse issues and 
resolve these in ways to achieve some desired goal or goals. Problems are all around 
us. The problems may vary in importance to the individual and the society, but have 
one common quality. A problem is a scenario which challenges us with the unknown. 
If a solution is known, it is no longer a problem. Initial engagement with a scenario 
is often to evaluate whether a problem actually exists. What may be a problem for 
one individual or in one context, may not be a problem for another—because they 
have different experiences, different knowledge, and different resources at hand. 
Problems are fascinating. Problems can present situations which lead to negative 
outcomes, or which can provide outcomes that help our societies progress. 

We vary in our ability to solve problems, and this has raised interest in the teach- 
ing and assessment of problem solving. Problems themselves vary widely, and this 
has led to interest in both domain-specific and domain-general application of 
problem-solving proficiencies. Do we use the same strategies or processes to solve 
a problem at home, or at school, or at work? Researchers who have identified sets of 
processes that need to be invoked to solve problems would answer in the affirma- 
tive. And it is this perspective that has fed twenty-first century interest in assessment 
of problem solving—with, for many jurisdictions, the consequent intention to 
improve citizens’ problem-solving abilities. 

Countries in East Africa, as well as globally, are looking to equip their young 
people with problem-solving competencies, which are then hoped to resolve the 
major problems that all societies confront. Problem solving is one of the competen- 
cies always included in any description of twenty-first century skills, and typically 
included in curricula developed by education systems. 

Since 2003, problem solving has been one of the constructs assessed in the 
OECD’s Programme for International Student Assessment (PISA), strengthening 
global knowledge of the construct within the confines of achievement that can be 
measured within the educational context. Notwithstanding the acceptance of a core 
definition and description of problem solving implied by the just under 80 jurisdic- 
tions and countries which participated in the 2018 PISA round, several countries 
have shown interest in their own formulation of the concept. For example, Molnar 
et al. (2022) report on how students from Jordan and Hungary approach problem 
solving differently, while Wicaksono and Korom (2022) draw on problem-solving 
frameworks predominantly developed or adapted for use in Indonesia. 

Problem solving in basic education systems has tended to be domain-specific. In 
other words, problems are posed to students within a particular domain, for example 
mathematics or science, and methods for resolving these problems are pursued 


4 Problem Solving in East Africa: A Contextualised Approach to Defining the Construct 49 


based on the knowledge domain required as well as a set of cognitive processes. The 
domain-specific approach to problem solving is best characterised by the work of 
Polya (1945), whose focus was on mathematics. Particularly in the twenty-first cen- 
tury, researchers have focussed on the complexity of problem solving as we face 
ill-defined problems (Funke, 2014), leading to more interest in domain-general 
applications of problem solving (Rudolph et al., 2017; Greiff et al., 2014), which are 
hoped to lead to building greater proficiencies. These problems are characterised by 
lack of definition, with goals unclear, and means of moving towards those goals also 
unclear. 

Increasingly, the domain-general conceptualisation of problem solving has 
become the target of international large scale assessment programs, including 
microworld situational scenarios (Greiff et al., 2015), and computerised tasks based 
on aspects of daily life included in PISA since 2012 (OECD, 2013). 

The distinction between domain-specific and domain-general approaches to 
problem solving is particularly relevant when considering how the education sector 
seeks to promote problem-solving competencies versus societal interest in the func- 
tioning of society and management of global issues. National curricula typically 
promote problem solving as it pertains to specific areas of learning, and mainly to 
mathematics and science. However, the current focus on problem solving highlights 
the anomaly between formal education’s traditional approach to the capability 
within mathematics and science studies, and society’s need for individuals to apply 
the steps of problem solving in daily life. Conceptualising problem solving as a 
twenty-first century skill is to prioritise its application in daily life, and to transfer 
use of steps or processes previously exercised within academic or research studies 
to practical issues. 

Another characteristic of problem solving is how it has increasingly been viewed 
as an information processing competency. It is therefore a competency that is more 
frequently described as a set of processes than as a skill. This characteristic sets it 
aside from many other competencies that fall under the twenty-first century skills’ 
or lifeskills’ umbrellas. The characteristic facilitates both our understanding of the 
competency and our ability to assess it. The Assessment of Lifeskills and Values in 
East Africa (ALIVE) project, designed to measure the lifeskills and values of ado- 
lescents, has benefited from this feature in that it provides the opportunity to formu- 
late the processes and find ‘carriers’ for these in daily life. And this is where we 
return to the need to ensure that the conceptualisation of the construct is true to its 
context. 


4.1.1 Conceptualising Problem Solving in East Africa 


The first task for ALIVE in developing tools for assessing the lifeskills and values 
of adolescents across Kenya, Tanzania, and Uganda, was to agree on commonly 
held understandings of these competencies in the participating countries. The devel- 
opment of conceptual and assessment frameworks by the ALiVE team was 
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contextualised through reference to recent research undertaken in the ALIVE par- 
ticipating countries. Giacomazzi et al. (2022) explored the nature of problem solv- 
ing across contexts, with a focus on its nature in three East African nations. Kenya, 
Tanzania, and Uganda in recent years have all integrated various lifeskills into their 
national education curricula (Kenya Institute of Curriculum Development, 2019; 
NCDC, 2019; Nkya et al., 2021), raising questions about reliance on Western defini- 
tions of these constructs and relevance of these constructs. 

Giacomazzi et al. (2022) describe problem solving as “identifying the problem, 
knowing and understanding the problem, asking for advice, evaluating the options 
and choosing between them, and finding the best solution” (p. 5). Many of the 
respondents in the contextualisation study associated problems and problem solving 
with interpersonal conflict or community difficulties. Asking for advice referred to 
seeking guidance from significant others and the community. Aspects of this con- 
ceptualisation differ from those used in most large-scale assessments and those 
associated with domain-specific competencies which prioritise cognitive elements 
only. A key element identified by Giacomazzi et al. (2022) is the social aspect of 
problem solving, and this of course draws attention to the role played by culture in 
our understanding of how problem solving takes place and is manifest. Incorporating 
this element is more complex than may initially appear. The consideration of com- 
munity and relationships rests on an individual’s knowledge of these aspects of 
human life. At an individual level, difficulty in problem solving occurs where lim- 
ited knowledge is available or when a situation itself is ambiguous. In extreme 
instances of such ambiguity, and particularly in cases where the problems are widely 
seen as critical, such problems have increasingly been been termed ‘wicked prob- 
lems’ (Funke et al., 2018). 

The ALiVE team that investigated problem solving also drew on the highly vis- 
ible framework used by OECD PISA (2013). That framework was set in principles 
well aligned with ALiVE’s current concerns with education and the strengthening 
the integration and development of these skills: “The acquisition of increased levels 
of problem-solving competency provides a basis for future learning, for effective 
participation in society and for conducting personal activities” (p. 120). The frame- 
work drew on an information processing approach, distinguishing it from domain- 
specific knowledge and approaches. The processes explored in PISA were: exploring 
and understanding; representing and formulating; planning and executing; and 
monitoring and reflecting. 

Tobinski and Fritz (2017) draw attention to problem solving occurring at the 
intersection of available and missing knowledge. They make the point that the initial 
state of a problem is based on previous knowledge. Although their interest was in 
handling of complex problems in the physical environment, the principle is useful 
in consideration of how the social context impacts problems and their solutions. 
Tobinski and Fritz (2017) draw attention to the initial difficulty and recognition of a 
problem being contingent on existing knowledge. If assessment of problem solving 
is to provide the same opportunities for any individual to demonstrate their problem- 
solving processes, then the scenario or situation which constitutes the problem 
needs to be equally familiar to all being assessed. 
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The significant challenges for ALIVE in development of an assessment for problem- 
solving skills of adolescents were: to accommodate the contextualisation recom- 
mendations inherent in Giacomazzi et al. (2022) in adoption of a conceptual 
structure with its consequences for an assessment framework; and to generate 
assessment tasks compatible with the limitations and affordances of household- 
based assessment. 

The ALIVE approach to design and development of the assessment was construct- 
driven (Wilson, 2005). This means that the form of the assessment tasks, coding 
criteria, and intended reporting style, are determined by the substance of the con- 
struct and how visible signs of this substance might best be captured. The theoreti- 
cal structure of the construct, problem solving, was therefore fundamental to the 
assessment design and form. 

Drawing upon research literature including sources quoted in this chapter, and 
the Giacomazzi et al. (2022) contextualisation study, the ALIVE initiative adapted a 
conceptual structure for problem solving and developed its assessment framework. 
The framework was developed through a realistic appraisal of what would be pos- 
sible to assess at household level. Figure 4.1 illustrates both the structure and 
framework. 

The first step in problem solving is to identify whether a problem exists. This 
step can be complex and will vary in cognitive demand according to the nature of 
the situation and the resources of the individual. A straightforward approach is to 
ask “who”, “what”, “how”, and “why”. The answers to these questions help to 
define the problem space; its current status and the ‘discomfort’ that characterises 
it; and the artefacts available within the space and those required upon which to 
draw (Care & Griffin, 2017). 

The conceptual structure shows the over-arching construct of problem solving, 
and three dimensions — defining the problem, finding the solution, and applying the 
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problem gathering problem solutions ee the solution solution the solution 


Fig. 4.1 Conceptual structure and assessment framework for problem solving 
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solution. Within each of these dimensions are sets of processes, or ‘subskills’. The 
subskills that could reasonably be assessed within the context of household-based 
assessment are identified in the assessment framework. These are shown in Fig. 4.1 
as shaded. The dimension of ‘applying the solution’ was not targeted in the assess- 
ment program, due to the logistical problems of enabling the activities. For each 
assessment task, ideas were created to formulate a scenario in which each of the 
problem-solving processes could be targeted. The ideas were generated by the 
ALiVE team and were drawn from community experiences. The ideas describe pos- 
sible scenarios in daily life; the assessment tasks comprised the scenario and a series 
of questions about it that would prompt responses interpretable within the problem- 
solving logic and its processes. 

The structure and the assessment framework adopted for the development of the 
household-based assessment draw on the OECD’s PISA problem-solving framework 
(2013) and the findings of Giacomazzi et al. (2022). The primary difference lies in 
how some of the cognitive processes are clustered together, and the inclusion of advice 
seeking. To some degree the differences are also a function of the generic levels at 
which processes are presented. For example, in the same way that ALiVE’s structure 
depicts dimensions and subskills within these, the OECD PISA example combines 
two distinct processes within what is shown in Fig. 4.2 as each of four steps. 


4.2.1 From Concept to Assessment Development 


In development of the assessment tool, three factors were considered. These were: 
the nature of the construct itself; the medium through which the assessment would 
be conducted; and the use to which the assessment results would be put. 


ALiVE 
recognising exploring selecting verifying 
information | understanding reflecting on implementing 
the alternative the the 
gathering the problem consequences the solution 
problem solutions solution solution 


Giacomazzi et al. (2022) 


knowing and asking choosing and 
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understanding the for finding the 
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OECD PISA (2013) 
representing and monitoring and 
exploring and understanding planning and executing 
formulating reflecting 


Fig. 4.2 Problem solving processes across conceptual structures 
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4.2.1.1 Nature of the Construct 


The adoption of the assumption that several distinct processes underly a problem- 
solving event, has implications for how assessment task scenarios must be created. 
Ideally, any given scenario will stimulate multiple processes, responses to which 
will be recordable and codable. Since a certain sequence in enactment of the pro- 
cesses typically occurs, both description of the task and the following questions and 
prompts must follow a logical order. The predictability of responses is a significant 
requirement in building the sequence. It means that each question and prompt must 
be phrased in such a way that the response can be interpreted within the parameters 
of the process for which the question or prompt has been created. 


4.2.1.2 Medium of Assessment 


The intention of ALiVE, as discussed in Shariff et al. (2024; Chap. 2, this volume) 
was to collect evidence of the competencies of 13—17 year old adolescents across 
Kenya, Tanzania, and Uganda regardless of whether they were attending formal 
education or not. Hence, the optimal way of accessing this group was through a 
household-based approach. This approach had implications for the administration 
of assessments, as well as the form of the assessment tools. 

The main features of the administration can be described briefly across practical 
matters, while these had implications for the nature of the assessment itself. It was 
essential that Test Administrators were fully versed in how to approach the adoles- 
cents, how to standardise the assessment interaction, and how to interpret responses 
so as to code these appropriately. In order to ensure this, the Test Administrators 
participated in three days of training, one day of which was dedicated to familiarisa- 
tion with the life skills and value to be assessed. The assessment of problem solving 
was to last for no longer than 7—10 min for each adolescent, with the prompts being 
provided orally on a one-to-one basis, with no stimulus aids. 

The implications of these features were several. 


e For orally administered assessment, lack of written or pictorial records or 
prompts meant that the assessment scenarios needed to be sufficiently simple 
and short that the task did not pose a significant short-term memory load for the 
adolescent; 

e Each assessment task needed to be communicable by the Test Administrator 
using simple language which could reasonably be comprehended by 13-17 year 
old adolescents across a range of backgrounds, education experience, and 
abilities; 

e Each assessment task needed to be capable of generating a predictable range of 
responses that would be amenable to immediate recording and coding by the Test 
Administrator; 
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e Each assessment task needed to have the potential to stimulate responses across 
several of the hypothesised problem-solving processes, in order to maximise 
time use efficiency; and 

e While drawing on situations familiar to the adolescent target group across the 
three countries the assessment tasks needed to be situated in contexts that would 
not discriminate between adolescents based on gender, religion, culture, or 
language. 


4.2.1.3 Use of the Assessment Results 


The intended use of the assessment results was for advocacy, not for reporting at 
individual adolescent level that could be used for instructional intervention. 
Accordingly, broad brush levels of competency and patterns across the adolescent 
target group in terms of gender, education, geographic location, language, would be 
the focus of reporting. Results could be framed in descriptive text that would clearly 
communicate what adolescents are able to do, so that these could be relayed to the 
education community. Given this anticipated use, it was important to generate infor- 
mation across a coherent but minimal number of subskills, to optimise messaging. 


4.3 Method 


4.3.1 Structuring the Work: Conceptualisation and Assessment 


The formal work sessions for defining problem solving and developing its assess- 
ment tools began in April 2021, with the first of six workshops attended by 47 rep- 
resentatives of the collaborating organisations. For each life skill and value, teams 
were assembled. The teams reviewed the findings of the contextualisation study and 
set the task of achieving consensus on a single definition for each construct. Drafts 
were presented for discussion across the four teams. This was followed by high- 
lighting sources for task ideas which led on to technical sessions on issues of fair- 
ness and difficulty levels. The ‘skills’ teams then took their work back to country 
convenings for further analysis. The second workshop reviewed status, checking 
likely alignment of task ideas against concept domains. This process continued 
through to July at which point ‘think-alouds’ (cognitive laboratories) were held to 
check early assumptions of the teams about the tasks, items, and scoring rubrics. By 
September of 2021, the analysis of student data from the think-alouds was used to 
set the assessment blueprints by identifying what was, and was not, possible to 
assess given the field conditions under which the large-scale assessments would be 
undertaken. From this time onward, each skill team worked independently of each 
other to fine-tune the assessments and the scoring rubrics. The problem-solving 
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team drew primarily upon members from Uganda.' Team membership fluctuated 
somewhat over the full development period. Analysis of dry run and pilot data early 
in 2022 was undertaken in a week long in-person workshop participated in by 
researchers.” This workshop assembled the evidence for finalisation of the large- 
scale assessment tools, confirmed construct structures, and subskills within con- 
structs, through statistical analysis of the response data. Within the workshop, 
separate teams were again convened to focus on the specific constructs. Analysis of 
data from the large-scale analysis was undertaken across October to November 2022.° 

With the conceptual structure and descriptions of the processes within this avail- 
able to them, the ALiVE problem-solving team first engaged in idea creation. 
Prompts for the process were to consider scenarios that adolescents might typically 
encounter, and evaluate whether these would provide the opportunity for display of 
the dimensions and their subskills. As the ideas were successively explored and 
critiqued, the elucidation of performance indicators and their quality criteria took 
place (Care et al., 2016). The first ‘check’ of the utility of the task ideas and their 
operationalisation into scenarios, prompts, and identification of performance indi- 
cators was undertaken through a cognitive laboratory process (Zucker et al., 2004; 
Griffin et al., 2012) with adolescents from the target population. Analysis of 
responses informed a review phase in which some task ideas were discarded, and 
for others their task descriptions, prompts, and performance indicators were refined. 
The next phase was a pilot of the assessment tasks with 392 adolescents (Kenya 
n= 154; Tanzania n = 79, Uganda n = 159). The pilot generated more information 
about time required for the assessment, need for modifications in phrasing of tasks, 
and refinement of coding criteria (informed by exploration of reliability across cod- 
ers). These pilot data were also used to explore the robustness of the dimensions and 
subskills scales. The final check included 308 Kenyan adolescents in a ‘dry run’ of 
the full assessment. The responses were collected by Test Administrators using tab- 
lets and the application Kobo Collect. These responses were then analysed to 
explore item response distributions, and variations across performance indicators. 
Subsequently, very minor amendments were made in phrasing, and some rubrics 
were redefined to ensure accuracy of coding. 

All problem-solving assessment tasks followed the same measurement design, 
with each task item generating one response for each subskill scale. From an initial 
pool of nine assessment tasks, six were good fit solutions, in the sense that these 
generated scales with excellent internal reliabilities. Of these six, just three were 
selected for the final large-scale assessment, in the interests of maintaining reason- 
able testing times for the adolescents. The responses given by adolescents to the 
tasks were coded according to criteria in the form of rubrics, contributing to the 
reporting for each subskill across three proficiency levels as illustrated in Table 4.1. 


'Kandi Alum, Martin Ariapa, James Droti, Mauro Giacomazzi, Joan Annet Kajura, 
Asha Kukundakwe, Mary Goretti Nakabugo, Florence Nansubuga, Faridah Nassereka, Godfrey 
Ntubiro, Saint Kizito Omala. 

*Problem solving participants: Martin Ariapa (Uganda), Daniel Marandu (Tanzania), Walter 
Odondi (Kenya). 
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Table 4.1 Subskills descriptions and performance indicators at increasing levels of proficiency 


Performance 
indicators 


Subskills descriptions 


Recognising the problem 


Levels of proficiency 


1 
Adolescent is... 


2. 
Adolescent is... 


3 


Adolescent is... 


This subskill targets an 
individual’s ability to 
explore and analyse a 
scenario to identify if 
there is a likelihood of a 


Unable to 
recognise key 
features of a 
situation that 
identify it as a 


Able to 
recognise a key 
feature of a 
situation that 
identifies it as 


Able to recognise 
multiple key 
features that 
imply there is a 
problem 


non-positive situation or | problem problematic 

outcome. 

Information gathering 

This subskill targets an | Unable to Able to identify | Able to identify 
individual’s ability to identify what one aspect or _ | different aspects 


think logically about 
what might have caused 
or contributed to a 


information is 
needed in order 
to solve the 


set of factors 
that, if known, 
might help to 


or sets of factors 
that, if known, 
might help to 


problem. problem solve the solve the problem 
problem 

Exploring solutions 

This subskill targets an | Unable to link | Able to identify | Able to 

individual’s ability to ideas about one main hypothesise 

link their knowledge of | nature of the approach to several possible 


the problem with 


problem with 


solving the 


solutions of 


possible actions or ways of dealing | problem different nature to 
solutions. with it solving the 
problem 
Selecting the solution 
This subskill targets an | Unable to Able to suggest | Able to suggest 
individual’s ability to suggest a a solution but is | and justify a 
evaluate multiple solution with not able to preferred solution 
possible solutions toa | justification for | justify why this | from among 
problem it is preferred several 


Each of the levels of proficiency for the performance indicators provided in 
Table 4.1, contributes to the over-arching identification of levels of performance for 
the over-arching skill, problem solving, which is reported in Ariapa et al. (2024; 
Chap. 10, this volume) 

Of interest is the degree to which this sequence model, running from recognition 
of problem to information gathering, then to exploring solutions prior to selection, 
is at odds with the cyclic notions of problem solving. Complex problem solving 
(Greiff et al., 2015; Bennett et al., 2003) typically refers to multiple cycles both 
across and within each of the separate subskills. This possibility was not allowed for 
in the ALIVE assessments, which instead followed a one-step logic within each 
process model. 
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4.3.2 Implications of a Contextualised Definition 
of Problem Solving 


The role played by the contextualised definition of problem solving was made 
explicit in the task idea creation and subsequent item development, and was demon- 
strated in response data gathered during the pilot phase. 

It is notable that the subskill of information gathering is the most difficult to 
perform by adolescents across Kenya, Tanzania, and Uganda (see Ariapa et al., 
2024; Chap. 10, this volume). The performance indicator at the highest level, ‘able 
to identify different aspects or sets of factors that, if known, might help to solve the 
problem’, is similarly difficult across all three assessment tasks. This demonstrates 
that the difficulty is not specific to task, but is associated with the subskill itself. It 
may be that this phenomenon is associated with the difference noted above between 
emphasis on cognitive processes seen in most conceptual structures for problem 
solving, rather than on relationships or community (Giacomazzi et al., 2022). The 
subskill requires that an individual hypothesise factors or elements that are part of a 
problematic situation, and then seek to inform these. This is essentially what is 
referred to elsewhere as exploring the problem space (e.g., Care & Griffin, 2017; 
Newell & Simon, 1972). If a first instinct when confronted with a problem is to seek 
advice from others (Giacomazzi et al., 2022), then this might impact on diverse 
approaches to information gathering. The issue can be seen through analysis of 
qualitative responses from adolescents to one of the tasks during the pilot process. 


4.3.2.1 Task Example 1 


One task scenario presents the issue of two school friends fighting. Data were col- 
lected during the pilot phase which informed the later coding of responses. The 
adolescent is asked ‘to solve this problem, what else do you need to know about it?’ 
From the qualitative responses collected in the pilot, approximately two thirds of 
adolescents respond that they need to know the cause of the fight; about one third 
respond that they need to know who started the fight or what the relationship is 
between the two friends. There is therefore little variation in responses, which dem- 
onstrates paucity in searching for other relevant artefacts or elements in the problem 
space. Questioning the reason for the fight implies that there might be some (pos- 
sibly community) value judgement about justification for the fight, while question- 
ing the relationship may address the point made in the contextualisation study—that 
relationships need to be prioritised. Whether these community or relationship pri- 
orities crowd out adolescents’ capacity to brainstorm other salient factors is a rea- 
sonable question. Whether both these main responses are symptomatic of reported 
comments collected in the contextualisation study is an interesting point. One young 
Kenyan is reported to have commented on problem solving: 
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Whenever he wants to solve a problem, he’s the person who does not ambush you wanting 
to solve the problem. He can’t be on one party’s side; he takes sides with both parties. If he 
takes sides with one person, he is not a problem solver. (Giacomazzi et al., 2022, p. 7) 


This limited variety of responses is noted in another task. The limitation may be due 
to widespread recognition of the type of risk described by the task, which therefore 
inhibits canvassing of more options. 


4.3.2.2 Task Example 2 


Another task scenario describes a younger sibling taking longer than anticipated to 
return from doing an errand. The interpretation of the scenario is based on the indi- 
vidual respondent’s knowledge of their community and the occurrences that typi- 
cally take place within it, reflecting Tobinski and Fritz’s (2017) point that pre-existing 
knowledge impacts on how an individual interprets any given event. In the pilot run 
for the assessments, just 6% of adolescents responded that there was no problem. A 
strong consensus for recognition of the problem at community level was indicated 
by the limited number of reasons proffered for the situation identified as a problem 
by the remaining 94%. Over one half identified ‘something bad’ including predomi- 
nantly kidnap, accident, and rape, while ‘child associated’ reasons such as getting 
lost, being distracted by friends, and playing, accounted for less than a quarter of 
reasons. Whether this particular set of reasons, in the proportions obtained, would 
have been proffered in other socio-economic, cultural or national environments is 
not known. However, the homogeneity of responses suggests a shared cultural 
understanding of most likely risks in the communities in which this adolescent 
group was based. It also sets the ‘difficulty’ level for the problem by virtue of estab- 
lishing this common knowledge base. 

The lack of variation in responses to need for additional information to inform 
the problem space may therefore be attributed to multiple factors, including the 
contextualised reality described above. A second may be the familiarity of the 
scenarios themselves on which the problems were based. Familiarity was a major 
criterion used to develop and then evaluate task ideas for problem setting (Tobinski 
& Fritz, 2017). Familiarity was judged necessary due to the household-based 
assessment approach that could not assume formal education content upon which 
to set problems. However, that familiarity itself could have nullified the need to 
think more creatively about the situation. And this takes us to the third factor, 
familiarity with an educational assessment paradigm. In all three countries, the 
experience of young people is that their responses to any assessment situation will 
be judged as either correct or incorrect—there is less scope for nuance. This may 
make it more likely that respondents will take a safe option of responding in con- 
ventional ways. 
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4.4 Discussion 


Adolescence is recognised as a time of growth across physical, cognitive, and 
social-emotional development (Erikson, 1968). It is a time when individuals take on 
significantly different roles from those which they have played in the past, and often 
significantly different from each other. Some adolescents remain primarily depen- 
dent on their family or carers while others run independent lives. Some are pre- 
pubescent while others are married or have offspring. To create assessment tools at 
the level of simplicity required for ALiVE while simultaneously engaging the inter- 
est of this target group poses some special problems. The difficulty for ALIVE was 
somewhat offset by the decision to use scenarios from daily life. Although individu- 
als at different levels of maturity might respond to these differently, the scenarios 
themselves would be similarly familiar to all. As shown in Ariapa et al. (2024; Chap. 
10, this volume) that assumption of responding at different levels of maturity was 
well-founded. However, the data also indicate that education has at least as great, if 
not more, influence on problem solving competencies in this population. If educa- 
tion has an impact on these competencies, it follows that problem solving is not 
purely a function of general intelligence nor of cognitive maturation. It means that 
instructional time in the context of a general curriculum is providing a learning 
environment in which problem-solving processes are nurtured. This can only be a 
positive for the goals of ALiVE and the education systems of the participating coun- 
tries—Kenya, Tanzania, and Uganda. 

The most known household-based assessments across Africa and Asia focus on 
literacy and numeracy (e.g., Uwezo, 2019; ASER, 2021). Such assessments of 
course are concerned with what is correct and what is not. In other words, accuracy 
is valued. This is typically well understood by children and adolescents, both in 
assessment at household level and in the school. The assessment of skills, however, 
cannot focus on accuracy since their application is about appropriateness rather than 
a binary judgement of success or failure; assessment therefore is about determining 
a level of competency. Responses are best interpreted as indicating either less or 
more of the target construct. Bringing an ‘accuracy mindset’ to an assessment event 
can be counter-productive when in fact the goal is to determine whether an appro- 
priate level of skill is brought to the demand. In the case of problem solving, its very 
nature requires generation of multiple hypotheses and solutions, and encourages 
trial and error. It is essentially an iterative process, and to judge just one step as 
accurate or inaccurate does not do justice to the overall process. 

Guidance to test-takers is generally provided before any assessment event, and 
so was the case with the ALIVE assessment. However, given lack of familiarity with 
this type of assessment, it is plausible that the adolescents might not have well 
understood the assessment goal of canvassing their thoughts, views, and processes. 
In addition, many adolescents were shy, and may have been inhibited in taking on a 
more interactive, conversational role in the assessment event. Giving short answers 
with the goal of providing a correct response may therefore have led to some 
under-performance. 
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The brevity of responses could also have been influenced by the framing of the 
task scenarios themselves. Since it was important to limit the cognitive load, given 
the lack of support materials, the stimulus text describing each scenario was kept as 
short as possible. This could have framed the mode of the adolescents’ responses by 
encouraging them to be equally brief. 

Two criteria applied to the creation of tasks were that responses would fall within 
a relatively limited range, and be compatible with coding. Formulation of tasks that 
fulfilled this curriculum could have cut out alternative scenarios that might have 
stimulated more varied or more creative responses, hence limiting the true range of 
competencies. The predictability was necessary, given the practicalities of the one- 
to-one assessment administration, the lack of additional stimulus materials, the rela- 
tively short training of the Test Administrators, and the need to record and code on 
the spot. However, evidence from the large-scale assessment results indicates that 
few adolescents were actually able to demonstrate high level competencies, so it is 
possible that this concern had an impact on this particular test event. It is important 
to consider for future development, whether artificial limits on opportunities to 
demonstrate high level competencies might be presented by such assessment 
approaches. 

The development and use of assessment tasks to capture demonstrations of 
problem-solving behaviour from adolescents in ALiVE was contextualised in three 
ways. The definition and structure of the construct was based on a study of under- 
standings of that construct. The task ideas were derived from events and circum- 
stances familiar in the lives of the adolescents. The environment in which adolescents 
demonstrated the skill was their home and community. Notwithstanding this locali- 
sation of the development, process, and use, the tasks functioned in a manner inter- 
pretable within existing knowledge about problem solving as researched widely 
(e.g., Csapó & Funke, 2017). This outcome is testament to the ubiquity of problem 
solving as a human process, the robustness of its core elements, and its adaptability 
under different circumstances. 
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Abstract The Assessment of Life Skills and Values in East Africa Study (ALIVE) 
focussed on four competencies with respect as the only selected value. This chapter 
explores the concepts of self-awareness and respect, delving into the process under- 
taken by ALiVE in the development of assessment tools to measure these constructs 
in East Africa. Self-awareness and respect are presented together due to some asso- 
ciations between them. Cultural influence on values such as respect is evident in 
worldwide perspectives, as well as in its local conceptualisation in East Africa. The 
chapter describes global and local perspectives on these constructs, and identifies 
the unique aspects of local conceptualisations in the East African region. The chap- 
ter discusses how self-awareness and respect are understood by adolescents, par- 
ents, and the communities in Kenya, Tanzania, and Uganda, and the implications of 
this on the development of the assessment tools. In addition, the assessment frame- 
works for both constructs are presented. These are expanded through description of 
the hypothesized proficiency levels and illustrated by sample tasks. 


5.1 Introduction 


Adolescents make up a population of 1.3 billion worldwide, with the sub-Saharan 
Africa population of adolescents expected to reach 500 million by 2050 (UNICEF, 
2022). Africa faces significant problems that cripple its economic growth. The 
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African continent has the youngest population in the world with 70% of youth in 
sub-Saharan Africa under the age of 30. It is therefore critical to build the skills of 
youth as a backbone to development of society and community. Youth unemploy- 
ment is one crisis rocking the economy. To empower youth, it is critical to nurture 
their key life skills and values as well as their technical work-related skills. Social- 
emotional skills such as self-awareness, and values such as respect, are critical to 
building of confidence, resilience, and interpersonal capabilities. 

In re-thinking the educational goals of today, scholars have argued for an educa- 
tion system that seeks to open the mind, nurture the spirit, and awaken the heart 
(Elmore, 2014). In response to this demand, the education systems of Kenya, 
Tanzania, and Uganda have adopted competence-based curricula that have inte- 
grated learning beyond the academic into education. Self-awareness is one such 
area of learning that has received attention because of its associations with cogni- 
tive development, and due to its influence on attainment of learning outcomes 
(Bryce et al., 2015). To Eysenck (1994), a deeper understanding of self is funda- 
mental not only to growth, but also to success in school and later in life. While 
educators are now being drawn into nurturing and developing these competencies 
in learners through a variety of experiences in and out of the classroom, there is 
need for educators to understand more about the domain of self-awareness. 

Regarding respect, East African countries have embraced values-based educa- 
tion as part of their curriculum reforms. In Kenya, for instance, the implementation 
of a new curriculum aims to nurture values such as love, responsibility, respect, 
unity, peace, patriotism, social justice and integrity within learners (KICD, 2019). 
The objective is to integrate these values into classroom instruction. However, like 
many other countries, East African countries face challenges in measuring the 
progress and effectiveness of values-based education. One of the reasons for this 
limitation is the lack of a systematic approach to evaluate the extent of values 
teaching in schools (Salleh et al., 2017). A related difficulty is the actual assess- 
ment of values, with researchers concerned about how to approach this, its validity 
and associated ethics issues (Forster, 2001). 

Self-awareness and respect are key elements of a person’s identity, attitudes 
toward others and society, and consequent capacity to interact with others. These 
concepts in traditional African and other cultures, and current perspectives about 
them, are brought together in this chapter. Each construct is considered separately 
to track their histories before drawing them together through their parallel assess- 
ment in the Assessment of Life Skills and Values in East Africa (ALiVE) initiative. 
The combination of the two constructs in this one review is intentional. There are 
interesting links between the two which imply some interdependence. Self- 
awareness is discussed first, following the logic of an argument that holds that 
awareness of self and others is a necessary recognition for the development of 
respect, for self, others, and the environment. 
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5.2 Review of Literature 


5.2.1  Self-Awareness 


Self-awareness is discussed as a prominent life skill, useful for improved work and 
life outcomes (Lawrence et al., 2018; Svalgaard, 2018). Studies have found that 
individuals with high levels of self-awareness tend to be more effective in the work- 
place. This is attributed to their ability to accurately evaluate their own capabilities, 
enabling them to adopt appropriate strategies for interpersonal interactions and task 
completion. Conversely, individuals low in self-awareness are less effective due to 
apparent misjudgement of personal competence and contributions (Carden 
et al., 2022). 

Self-awareness is believed to enhance decision making, team performance, and 
authentic leadership (Carden et al., 2022). Organizations increasingly rely on teams 
to carry out complex tasks. Teams are comprised of individual members whose 
unique characteristics, including knowledge, skills, attitudes, beliefs, dispositional 
tendencies, and behaviours, collectively shape the team’s overall capacity 
(Kozlowski & Klein, 2000). Despite the extensive body of literature on teams and 
team-member characteristics, the role of self-awareness is relatively un-explored. 
This is despite widespread recognition that individual contributions are crucial for 
team-level functioning (Dierdorff et al., 2019). 

In terms of life outcomes, self-awareness has been recognized by practitioners 
and researchers as a crucial tool for reducing psychological distress and as a path- 
way to personal growth for individuals. Various aspects of self-awareness, such as 
mindfulness and rumination, play a mediating role in the impact of mindfulness- 
based interventions on mental health outcomes (Gu et al., 2015). Nevertheless, the 
significance of self-awareness extends beyond well-being and mental health to sig- 
nificant effects on day-to-day functioning. It has implications for performance, as 
reflection and mindfulness promote persistence in tasks despite performance-related 
stress, while rumination is associated with interpersonal difficulties (Sutton, 2016). 

The literature on self-awareness often fails to acknowledge the complexity of the 
construct (Carden et al., 2022; Sutton, 2016; Sutton et al., 2015). It is frequently 
conflated with self-consciousness and self-knowledge, leading to confusion and 
inconsistency in understanding. According to CASEL (2023), people are self-aware 
when they can accurately recognize their own emotions, thinking, and values and 
how these determine how they behave. Dierdorff et al. (2019) define self-awareness 
in regards to team performance as an individual’s ability to assess and understand 
what one offers behaviourally to the team. Morin (2011) considers self-awareness 
as the ability to become the object of one’s focus by looking at oneself from others’ 
viewpoints, thereby creating an imagined distance between the individual and the 
self. Goleman (2006) affirms that self-awareness is knowing one’s emotions (emo- 
tional intelligence), which Salovey and Mayer (1990) define as “the subset of social 
intelligence that involves the ability to monitor one’s own and others’ feelings and 
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emotions, to discriminate among them and to use this information to guide one’s 
thinking and actions” (p. 189). 

These perspectives and interpretations of self-awareness (Sutton, 2016) make 
complicated theory development in education, the establishment of accurate mea- 
surement tools, and progression of research (Carden et al., 2022). This complexity 
is not surprising when considering the broader notions of self and of awareness. Self 
is subject to diverse philosophical perspectives (Bachkirova, 2011). Firstly, there is 
the social behaviourism view, which considers the self in relation to social processes 
and communication (Carden et al., 2022). According to this perspective, the devel- 
opment of self is influenced by observing and interacting with others (Baumeister, 
2005). Secondly, there is the intrapersonal perspective that posits the existence of 
multiple layers and dimensions within the self (Harter, 2006). These dimensions can 
be both conscious and unconscious (Bachkirova). Therefore, the key distinction 
between these two perspectives lies in whether the self is understood in relation to 
others, or whether it is focused on the various layers and dimensions within oneself, 
as emphasized in the multidimensional layered view. More frequently, it is argued 
that the self is inherently multidimensional, comprising conscious and unconscious 
layers, and is influenced by observations of others. 

The concept of awareness alone also presents a somewhat perplexing picture. 
Awareness is often used interchangeably with consciousness (Fromm, 1965; 
Vaneechoutte, 2000) and psychological mindedness (Beitel et al., 2005). The litera- 
ture on awareness can be categorized into three core concepts (Carden et al., 2022). 
First, cognitive awareness emphasises an individual’s comprehension of their own 
perception and thinking, with awareness representing the capacity to gain a thor- 
ough and accurate understanding of these processes. Second, there is a perspective 
that asserts awareness to be multilevel, encompassing both conscious and uncon- 
scious aspects, culminating in a stage of awareness where individuals process their 
physical and mental experiences (Vaneechoutte, 2000). The third conceptualisation 
considers awareness in relation to recognising and understanding the emotions and 
feelings of others, taking into account one’s impact on them (Beck et al., 2004). 

Given the lack of consensus regarding self and awareness, it is not surprising that 
the self-awareness construct is also unclear (Sutton et al., 2015). Initially, the con- 
cept of self-awareness was perceived to have two dimensions (Duval & Wicklund, 
1972). The first dimension is subjective self-awareness, which refers to a state of 
consciousness where attention is focused on external events. The second dimension 
is objective self-awareness, which involves a concentrated focus on the self. This 
two-dimensional approach suggests that self-awareness is achieved by directing 
attention towards oneself and comparing one’s actions to self-established standards. 
This comparison can lead to an aversive state if a discrepancy arises between self- 
perception and self-developed standards, resulting in a negative state of mind (Silvia 
& Duval, 2001). This perception of self-awareness as an aversive state is associated 
with the rumination aspect of self-consciousness (Trapnell & Campbell, 1999), 
which tends to focus on negative thoughts such as past mistakes or feelings of inad- 
equacy, potentially leading to mental health issues. However, there is an alternative 
perspective that distinguishes between rumination and reflection in which the latter 
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is considered a positive process that fosters self-consciousness and facilitates learn- 
ing (Carden et al., 2022). 

A recent systematic review of the literature on self-awareness shows a 
hierarchically-structured conceptualisation of self-awareness (Fig. 5.1). The main 
components of the construct of self-awareness are shown as either intra- or 
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inter-personal components. The intrapersonal aspect focuses on an individual’s 
awareness of their own resources and internal state of mind, while the interpersonal 
aspect centres on an individual’s awareness of their impact on others. Across the 
two aspects, Carden et al. (2022) identified seven separate components: (i) beliefs 
and values: personal attitudes and important concepts that individuals introspec- 
tively explore to understand their behaviour and reactions; (ii) internal mental state: 
includes cognition and emotions; (iii) physical sensations: refers to physiological 
responses or bodily reactions; (iv) personality traits: awareness of one’s character 
traits, strengths, and weaknesses; (v) motivations: personal drivers or reasons for 
behaviour; (vi) behaviours: actions that others observe and can interpret, thereby 
affecting interpersonal interactions; and (vii) others’ perceptions: awareness of how 
one is perceived by others, often involving feedback from others. 

These components enable individuals to understand their behaviour and choices 
and ultimately develop self-awareness. Introspective exploration and objective 
examination of these components are emphasized in the literature. 


5.2.2 Respect 


Respect is recognized as an essential socio-emotional virtue in everyday life. Its 
central role in shaping and guiding meaningful human interactions and for living 
quality, morally upright lives is a point of emphasis (Dillon, 2003; Malti et al., 
2020). They identify it as important for forming and maintaining human relation- 
ships, while being complex, contextual, and multidimensional in nature. While a 
disparity exists in literature as to what constitutes respect, three defining features of 
respect are mentioned; attention—an engagement with the object in order to fully 
recognize it; consideration—examining the object; and deference—the acknowl- 
edgement of the different social positions. 

Respect can be regarded as manifest through the valuing, holding in high esteem, 
admiring, or honouring the good and moral features of oneself and others (Dillon, 
2003; Malti et al., 2020). The distinction between such positive positions and visible 
signs of respect, particularly where obedience or status implies particular ways of 
behaving, complicates understandings of respect. Feeling, or not feeling, respect for 
something or someone is not necessarily demonstrated by behaving in a culturally 
or socially respectful way (e.g., Arrington, 1978). 

From a philosophical perspective, respect is considered as the fair and equal 
treatment of all humans with dignity for the fact that they are humans (Dillon, 2003; 
Malti et al., 2020). Dillon asserts that all humans capable of rational action are owed 
unconditional respect because of their intrinsic worth and value. In this context, 
respect entails expressing consideration of another person to fully recognize her or 
him (Malti et al., 2020). This is expanded upon in that respect can be shown through 
silently observing in order to evaluate what the other needs (Lawrence- 
Lightfoot, 2012). 
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According to OECD (2018) respect is one of the most frequently cited values in 
the education curricula and “it concerns the capacity of one to value oneself, others, 
and the environment we are all in, and to give due regard for the feelings, wishes, or 
rights of self and others as well as those surrounding us that may not express wishes 
(e.g., environment, animals). Respect is demonstrated through behaviour and com- 
munication which will vary based on cultural context. Respect for cultural diversity, 
for example, means valuing the many differences and similarities of others that may 
be present. Respect for nature involves environmental ethics” (pp. 25—26). Respect 
can thus be seen as multidimensional since it lies in awareness and honouring of 
self, others, and the surroundings. 

Respect for self is defined as an evaluation of one’s personhood, rights, character, 
status, and achievements, including self-appraisal of one’s talents, accomplish- 
ments, and excellencies (Seglow, 2016). It has three components (i) self-worth (the 
value that one attaches to his/her attributes, talents, skills, abilities, and strengths, 
which allows the person to act in a way worthy of admiration and to stand for what 
he/she beliefs on); (ii) self-acceptance (the ability of one to realise one’s mistake, 
forgive oneself, and appreciate one’s weakness); and (iii) self-love (the ability of 
one to reduce harm to self, take care of and appreciate oneself unconditionally). 
According to Harter (2006), self-worth is the awareness of good possessed by the 
self and refers to the overall appraisal of one’s worth or value as a person. This 
shows how self-worth and self-awareness are intertwined since self-awareness 
allows one to have a deeper understanding of their inner being. According to Hibbert 
(2015), self-awareness is the foundation of self-worth. She argues that one cannot 
achieve self-worth without being self-aware. Self-love on the other hand is a gentle 
love for self where one strives to cultivate care for self and development (Benjamin 
et al., 2006). According to this view, people with self-love will do no harm to self 
(either body or reputation), and will always work to overcome their weaknesses, 
leading to achieving self-worth. Scholars such as Lin et al. (2003) and Huang et al. 
(2018) agree with this idea of self-love as the ability to cherish one’s character and 
pay attention to one’s words, ability, and deeds. 

A second dimension of respect in the literature relates to respect for others. 
Respect for others includes acknowledging peoples’ capability, not hurting others 
intentionally, paying attention when others speak, affirming others’ qualities, 
treating others equitably, treating with consideration and treating people equally 
(irrespective of their race, tribe, culture, religion, age, social, economic status, 
physical ability, and abilities, regardless of preferences) (Wood, 2009). Other 
indicators include acknowledging personal and physical boundaries and not 
invading others’ privacy, appreciating other’s perspectives/contributions and 
points of view, valuing others’ time and property, and avoiding hasty judgments 
when reacting to people. 

The third dimension is respect for surroundings—place or environment. The 
question of whether the value of respect should be extended only to rational beings 
or to other entities as well has sparked debate. Critics of Kant’s ethical theory argue 
that by emphasising the moral worth of rational beings and treating non-persons as 
mere means to their ends, supports the domination and exploitation of non-persons 
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and the natural environment. Nevertheless, other interpretations of Kantian ethics 
affirm that respect for persons logically entails respect for non-persons. It suggests 
that not only should humanity in persons be respected, but also things that bear 
certain relationships to rational nature. Moral agents are recognised as having the 
highest moral standing and worth while acknowledging that other beings have lesser 
but still morally significant standing and worth, deserving some degree of respect. 
This perspective allows for the justifiable use of non-persons while respectfully 
acknowledging their moral worth. Extensive philosophical work, particularly in 
environmental ethics, explores the practical implications of extending respect to 
entities beyond persons. 


5.3 Self-Awareness and Respect in East Africa 


In 2018, the Regional Education Learning Initiative (RELD, through the Values and 
Life Skills (VaLi) thematic group, proposed a collaborative effort with local leaders 
in Kenya, Tanzania, and Uganda to develop assessments tailored to the context. The 
initial phase involved conducting ethnographic interviews in various locations, 
including rural and urban areas, within these countries. These interviews, carried 
out in November 2020, targeted adolescents, parents, and key individuals such as 
teachers, social workers, youth patrons/matrons, and religious leaders. The purpose 
of the study was to gather data on participants’ perceptions and understanding of 
specific ALIVE competencies. To achieve this, an ethnographic design was 
employed to explore and capture participants’ perspectives and comprehension of 
self-awareness and respect in the contexts of the participating countries (Ariapa & 
Akongo, 2021; Serwanga & Atuheire, 2021). 

The second phase involved the development of contextualised assessments of 
self-awareness and respect through a series of workshops and collaborative activity. 
Formal work sessions began in April 2021, with the participation of 47 representa- 
tives from collaborating organizations. Teams were formed to review the findings of 
the contextualisation study and so to establish consensus on definitions of self- 
awareness and respect. The phase continued with checking the assessment tools at 
multiple stages to test assumptions about the tasks, items, and scoring rubrics. Each 
skill team worked independently to refine the assessments and scoring rubrics. 
Adolescent responses to the assessment tasks were collected through dry run and 
pilot events. These data were analysed to explore how they cohered with the hypoth- 
eses that had been formed by the skills teams about the nature and structure of each 
of the constructs. 

The third phase saw the implementation of large-scale assessment across Kenya, 
Tanzania, and Uganda. The assessment was implemented at household level in 
order to capture the proficiencies of adolescents both in and out of school, and in 
and out of employment (Nakabugo et al., 2024; Chap. 8, this volume). This was fol- 
lowed by analyses of the data from over 45,000 adolescents to generate results that 
were then disseminated in each country (Ariapa et al., 2024; Chap. 10, this volume). 
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The assessment tools were developed to cohere with the conceptualisations of 
self-awareness and respect as identified through the contextualisation study. The 
two teams dedicated to the development of assessment tasks for self-awareness 
and respect developed scenarios to address these constructs and selected subskills. 
The scenarios were based on community experiences and described everyday life 
situations. Each assessment task comprised a description of a scenario and a series 
of questions that prompted responses that would reflect behaviours aligned with 
the subskills (Mutweleli et al., 2024; Chap. 9, this volume). The scenarios were 
designed to prompt responses that could be evaluated according to hypothesised 
levels of proficiency. 

The following sections describe the contextualised structure for each con- 
struct, identify which subskills of the constructs were targeted for assessment, 
and tabulate descriptions of the levels of proficiency at which adolescents could 
respond when their self-awareness and respect behaviours and perceptions were 
prompted. 


5.3.1  Self-Awareness in ALIVE 


The contextualisation study found that self-awareness is mostly referred to as the 
ability to understand and know oneself (Serwanga & Atuheire, 2021). Self- 
knowledge involves a clear understanding of one’s rights, identity, emotions, prefer- 
ences, abilities, strengths, and weaknesses, as well as desires and goals. For some 
participants in the contextualisation study, self-awareness went deeper, determining 
how one behaves towards teachers, parents, and peers in the community. However, 
some participants limited self-knowledge to the understanding of one’s surround- 
ings or environment, without considering internal reflection and evaluation. 

Another aspect that emerged in the definition of this skill is the notion of taking 
care of oneself. Participants highlighted that a self-aware individual makes careful 
choices regarding health and psychological well-being. This includes proper nutri- 
tion, medication adherence, and awareness of how one’s behaviour can impact well- 
being. Figure 5.2 summarises the skill structure of self-awareness as it emerged 
from the contextualisation study and identifies the dimensions or subskills that were 
assessed. 

The conceptual structure of self-awareness consists of two dimensions: internal 
self-awareness and external self-awareness. These dimensions resonate with the 
framework developed by Carden et al. (2022). Each dimension contains subskills. 
Taking into consideration the affordances of household-based assessment, those 
subskills that were targeted for assessment were self-management and perspective 
taking (shown shaded in Fig. 5.1). 

The behaviours exhibited by the adolescents can be understood and evaluated 
based on the proficiency levels in Table 5.1. These levels represent increasing 
levels of sophistication and complexity as the adolescents demonstrate their 
abilities. 


Fig. 5.2 Self-awareness 
skill structure 
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Table 5.1 Qualitative descriptors of self-awareness levels of proficiency 
Level 1 Level 2 
Adolescent Adolescent Level 3 Level 4 
Construct 18... is... Adolescent is... | Adolescent is... 
Self-management: This | Unable to Able to control | Sufficiently Sufficiently 
subskill targets an regulate self ina self-aware and | self-aware and 
individual’s ability to negative negative or confident in self | confident to 
recognize and express emotions or stressful to respond respond 
emotions, to assess self, | responses situation adaptively to adaptively even 
to reflect, and to manage through most situations | when directly 
emotions. repression of confronted or 
emotion or attacked 
avoidance 
Perspective taking: Aware of Aware that Aware that 
This subskill targets an | others’ others may be | others act on the 
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behave the way they do _| to oneself personal and 
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The proficiency levels in Table 5.1 are stated as performance indicators so that 
there is a direct link to behaviours or perceptions prompted by the tasks and 
phrased in terms aligned with self-awareness. The assessment for self-awareness 
consists of five task scenarios with a subset of 12 items. The items in these tasks 
assess two subskills of self-awareness: self-management, which involves manag- 
ing emotions and stress, and perspective taking, which involves understanding 
others’ views and actions, adjusting to them, and recognising one’s own identity 
and position in the family, society, and community. 

As an example, one task scenario prompted adolescents to reflect on the possi- 
bility of parents not providing an anticipated gift: “Your parents told you they are 
going to give you a bicycle for your birthday to help you go to school, and you have 
excitedly told your friends. When the day arrives, your parents do not give you the 
bicycle.” Several follow up questions were asked to understand the ability of the 
respondent to see somebody else’s perspective, and to self-manage their own 
response. Adolescents were asked: “How will you react to your parents? And 
why?”; “If you were one of the friends, how would you react?”; and “Your friends 
are laughing at you, how would you react?” 

Disappointment and anger were the most frequent responses from adolescents 
in the pilot study. Quite frequent also was reference to the parents having lied, 
which is not actually described by the scenario. It is striking that only a few said 
that they would “feel bad but have to understand the reason behind.” Concerning 
them being the object of derision by friends, most adolescents expressed sadness 
or disappointment. Very few took a conscious decision to confront the matter with 
friends through a discussion that would have respected both the motivations of the 
parents and the disappointment felt by the adolescent. 


5.3.2 Respect in ALIVE 


The contextualisation study (Ariapa & Akongo, 2021) found that valuing others was 
identified as the primary definition of respect. Valuing others involves honouring, 
taking care of others, considering them important and useful, and treating adults and 
elders with dignity. The study also revealed that respect has meanings variously 
influenced by culture and prevailing social norms. A comprehensive understanding 
of respect and its expression requires familiarity with one’s community and its lan- 
guage, and the ability to navigate within its social conventions to ensure acceptable 
and respectful interactions. Aspects such as self-care, self-awareness, and being a 
good person, also emerged as defining elements of respect. 

According to most participants interviewed, respect is closely linked to one’s 
behaviour and conduct. Actions such as greeting others, appropriate dressing, obe- 
dience, refraining from wrongdoing, discipline, and demonstrating exemplary char- 
acter are all seen as manifestations of respect. 
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Respect for others, from the East African perspective (Ariapa & Akongo, 2021), 
encompasses the dimension of regarding and valuing others. This dimension is 
reflected through acknowledging the capabilities of others, refraining from inten- 
tionally causing harm, actively listening, and treating others with fairness and 
equity. Indicators of respect for others include respecting personal and physical 
boundaries, refraining from invading others’ privacy, and appreciating the perspec- 
tives and contributions of others. Figure 5.3 illustrates the structure of the respect 
construct based on findings from the contextualisation study. 

The conceptual structure of respect as a value is aligned with findings from the 
literature. Specifically, self-respect can be described through three main dimensions 
(Dillon, 2003): self-respect, respect for other people, and respect for environment or 
animals. Each of these dimensions has associated subskills. The subskill, respect for 
others, was directly targeted for assessment. For design of the scenarios, the team 
searched for generic yet realistic social situations with which all participants could 
be assumed to be familiar. They were as simple and concise as possible, providing 
sufficient context to stimulate an authentic set of responses. Their perspectives on 
respect for others are indicated by how they responded to the scenarios; Table 5.2 
describes lower to higher understanding of the implications of respect on daily life 
and behaviour of the individual. 


Respect 


Respect for 


Respect for Self Respect for others . 
environment 


Self worth Regard for others Appreciating places 


Taking care of man- 
Self love made and natural 
resources 


Self acceptance 


Fig. 5.3 Respect skill structure 
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Table 5.2 Qualitative descriptors of respect levels of proficiency 


Level 1 
Adolescent | Level 2 Adolescent | Level 3 Adolescent 
Construct | is... ISe ESS Level 4 Adolescent is... 
Respect | Unable to Aware of | Able to interpret bad | Aware of links between 
|respondina |infringement of behaviour as lack of | respect for property and 
| relevant way. | rights, or of bad respect for others or | respect for person, and 
| behaviour by one self, and may take will act in a respectful 
person toward conciliatory steps to | way toward others and 
| another but does not | resolve situations. | in defence of others and 
‘call it out’. | | self. 


The assessment of respect includes four task scenarios with a subset of 10 items. 
All four scenarios follow a similar pattern, starting from awareness of a lack of 
kindness to recognising actions that indicate a lack of respect. The items in the 
assessment evaluate the adolescent’s consideration for others and respect for prop- 
erty. They assess awareness of infringing on others’ rights, recognition of personal 
wrongdoing, respect for the rights of others, and willingness to take responsibility 
for one’s actions. 

One task scenario used in the assessment was described as follows: “Amani’s 
parents went to a wedding and left her with the younger sister. The younger sister 
got sick, and Amani had to take her to the health centre. When they got to the health 
centre, Amani went straight to the doctor’s room, bypassing other patients even 
though her sister was not very sick.” 

This scenario provides a situation in which the needs of the self should be evalu- 
ated in the context of the needs of others. There are two main steps. First, the ado- 
lescent must consider if their own needs pre-empt those of others; and second, if the 
decision is that their needs are pre-emptive, then the adolescent must consider how 
to negotiate their primacy. This negotiation would indicate an acknowledgement of 
the rights of others, an indicator of respect based on personhood. 

Of the responses given by the adolescents in the pilot, around 15% of the adoles- 
cents showed a lack of awareness of the actual problem caused by Amani. They did 
not express any concern about the situation, or they responded in ways inconsistent 
with the actual issue. However, the vast majority affirmed that they would have told 
Amani to respect the queue or would have recommended that she ask permission to 
skip it. Taking on the perspective of the patients waiting in the queue, the majority 
responded that they would have felt disrespected, and angry about Amani’s behav- 
iour. These responses were quite homogeneous in nature, perhaps reflecting similar 
understandings of a familiar situation, as was the intention in the design of the sce- 
nario tasks. 
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5.4 Discussion 


The findings of the contextualisation study (Serwanga & Atuheire, 2021) revealed 
that self-awareness in East African countries refers to a deep understanding of one- 
self in terms of rights, health, feelings, preferences, abilities, and values that influ- 
ence decision making. It is a dynamic and continuously evolving process of 
self-discovery that never achieves saturation. The structure of self-awareness devel- 
oped by the ALIVE team showed that this skill consists of two main dimensions: 
internal self-awareness (including the subskills of emotional recognition and expres- 
sion, self-management, and assessment) and external self-awareness (essentially the 
skills that inform perspective-taking). These findings align with existing literature 
(Carden et al., 2022; Lawrence et al., 2018; Morin, 2011). 

In terms of contextualising respect, Ariapa and Akongo (2021) highlighted the 
most common perception of respect as revolving around honouring and valuing oth- 
ers. Behavioural examples of these values include appropriate attire, showing 
respect to elders through greetings, obedience to authority, humility, and a sense of 
responsibility. The study participants described a respectful young person as some- 
one who embodies kindness, friendliness, a willingness to learn from others, and a 
sense of responsibility and accountability towards others. The main dimensions of 
respect were respect for self, respect for others, and respect for the environment. 

Self-awareness and respect are both rooted in an awareness of one’s own worth, 
moderated by internal and external factors. Both constructs concern the need to 
focus and reflect on one’s inner processes and experiences as well as being aware of 
others’ perceptions and experiences (Sutton, 2016). Both constructs focus on the 
interplay between the self and its surroundings (people and environment) such that 
a proper understanding of oneself not only leads to self-love and self-acceptance, 
but also to deeper understanding of the social environment within which the self is 
located. In the ALIVE framework, this is seen is the dimensions of respect for self, 
others, and environment, while it translates to internal and external self-awareness 
in the skills structure of self-awareness. 

This linkage is important because it explores self-awareness as foundational to 
respect. One cannot achieve respect (for self, others, or environment) without appre- 
ciable levels of self-awareness. Self-worth, which is a dimension of respect, is 
described as the value that one attaches to one’s own attributes, talents, skills, abili- 
ties, and strength. These, in turn, allow one to act in a way worthy of admiration and 
to stand for what one believes in, intertwined with one’s awareness of self. 

This chapter described the process of developing contextually appropriate tools 
for measuring self-awareness and respect. Prior to this time, in the East African 
context, the assessment of self-awareness and respect relied on tools that were 
developed in other countries and not contextualised. The chapter emphasized the 
importance of ensuring a contextually appropriate definition of self-awareness and 
respect to guide development of assessment tasks that allow for measuring skills 
and values and tracking progress. 


5 Self-Awareness and Respect in East Africa: A Contextualised Approach to Defining... 77 


References 


Ariapa, M., & Akongo, S. R. (2021). The understanding of respect in Kenya, Tanzania, and 
Uganda. A report for the Assessment of Life Skills and Values in East Africa (ALiVE) project. 
RELI. https://reliafrica.org/alive/ 

Ariapa, M., Pavlovic, M., & Care, E. (2024). Measuring adolescents’ life skills and values: Method 
and results from East Africa. In E. Care, M. Giacomazzi, & J. K. Mugo (Eds.), The contextu- 
alisation of 21st century skills: Assessment in East Africa. Springer. 

Arrington, R. L. (1978). On respect. Journal of Value Inquiry, 12(1), 1-12. 

Bachkirova, T. (2011). Developmental coaching: Working with the self. McGraw-Hill Education. 

Baumeister, R. F. (2005). The cultural animal: Human nature, meaning, and social life. Oxford 
University Press. 

Beck, A. T., Baruch, E., Balter, J. M., Steer, R. A., & Warman, D. M. (2004). A new instrument 
for measuring insight: The Beck cognitive insight scale. Schizophrenia Research, 68(2-3), 
319-329. https://doi.org/10.1016/S0920-9964(03)00189-0 

Beitel, M., Ferrer, E., & Cecero, J. J. (2005). Psychological mindedness and awareness of self and 
others. Journal of Clinical Psychology, 61(6), 739-750. https://doi.org/10.1002/jclp.20095. 

Benjamin, L. S., Jeffrey, C. R., & Kenneth, C. L. (2006). The use of structural analysis of social 
behavior (SASB) as an assessment tool. Annual Review of Clinical Psychology, 2, 83-109. 
https://doi.org/10.1146/annurey.clinpsy.2.022305.095337 

Bryce, D., Whitebread, D., & Szücs, D. (2015). The relationships among executive functions, 
metacognitive skills and educational achievement in 5 and 7 year-old children. Metacognition 
and Learning, 10(2), 181-198. https://doi.org/10.1007/s 1 1409-014-9120-4 

Carden, J., Jones, R. J., & Passmore, J. (2022). Defining self-awareness in the context of adult 
development: A systematic literature review. Journal of Management Education, 46(1), 
140-177. https://doi.org/10.1177/105256292 1990065 

Collaborative for Academic, Social, and Emotional Learning. (2023, May 27). CASEL’s SEL 
framework. https://casel.org/casel-sel-framework- 1 1-2020/?download=true 

Dierdorff, E. C., Fisher, D. M., & Rubin, R. S. (2019). The power of percipience: Consequences of 
self-awareness in teams on team-level functioning and performance. Journal of Management, 
45(7), 2891-2919. https://doi.org/10.1177/01492063 18774622 

Dillon, R. (2003). Respect. In E. Zalta & U. Nodelman (Eds.), Stanford encyclopedia of philoso- 
phy. The Metaphysics Research Lab. https://plato.stanford.edu/entries/respect/ 

Duval, S., & Wicklund, R. A. (1972). A theory of objective self-awareness. Academic. 

Elmore, K. C. (2014). Improving academic effort and achievement among low-income minority 
youth using small scale interventions. University of Michigan. https://deepblue.lib.umich.edu/ 
bitstream/handle/2027.42/108857/kelmore_1.pdf?sequence=1 

Eysenck, H. J. (1994). Creativity and personality: Word association, origence, and psychoticism. 
Creativity Research Journal, 7(2), 209-216. https://doi.org/10.1080/104004 19409534525 

Forster, K. (2001). The assessment of values in schools. Journals of Beliefs & Values, 22(1), 31—40. 
https://doi.org/10.1080/1361760120039220 

Fromm, E. (1965). Awareness versus consciousness. Psychological Reports, 16(3), 711-712. 
https://doi.org/10.2466/pr0.1965.16.3.711. 

Goleman, D. (2006). The socially intelligent. Educational Leadership, 64(1), 76-81. 

Gu, J., Strauss, C., Bond, R., & Cavanagh, K. (2015). How do mindfulness-based cognitive therapy 
and mindfulness-based stress reduction improve mental health and wellbeing? A systematic 
review and meta-analysis of mediation studies. Clinical Psychology Review, 37, 1—12. https:// 
doi.org/10.1016/j.cpr.2015.01.006 

Harter, S. (2006). Handbook of child psychology, social, emotional, and personality development 
(Vol. 3, 6th ed.). Wiley. 

Hibbert, C. G. (2015). Who am I without you?: Fifty-two ways to rebuild self-esteem after a 
breakup. New Harbinger Publications. 


78 P. Ngina et al. 


Huang, Z. H., Jing, Y. M., Yu, F., Gu, R. L., Zhou, X. Y., Zhang, J., & Cai, H. (2018). Increasing 
individualism and decreasing collectivism? Cultural and psychological change around the 
globe. Advances in Psychological Science, 26(11), 2068-2080. https://doi.org/10.3724/ 
SP.J.1042.2018.02068 

Kenya Institute of Curriculum Development. (2019). Basic education curriculum framework. 
https://kicd.ac.ke/curriculum-reform/basic-education-curriculum-framework/ 

Kozlowski, S. W., & Klein, K. J. (2000). A multilevel approach to theory and research in organiza- 
tions: Contextual, temporal, and emergent processes. In K. J. Klein & S. W. Kozlowski (Eds.), 
Multilevel theory, research, and methods in organizations: Foundations, extensions, and new 
directions (pp. 3-90). Jossey-Bass/Wiley. 

Lawrence, E., Dunn, M., & Weisfeld-Spolter, S. (2018). Developing leadership potential in gradu- 
ate students with assessment, self-awareness, reflection and coaching. Journal of Management 
Development, 37(8), 634-651. https://doi.org/10.1108/JMD-11-2017-0390 

Lawrence-Lightfoot, S. (2012). Respect: On witness and justice. American Journal of 
Orthopsychiatry, 82(3), 447-454. https://doi.org/10.1111/j.1939-0025.2012.01174.x 

Lin, C., Yang, Z., & Huang, X. T. (2003). Psychology dictionary. Shanghai Education Press. 

Malti, T., Peplak, J., & Zhang, L. (2020). The development of respect in children and adolescents. 
In G. A. Moore (Ed.), Monographs of the society for research in child development (Vol. 85, 
pp. 7-99). The Society for Research in Child Development. https://srcd.onlinelibrary.wiley. 
com/doi/pdf/10.1111/mono.12417 

Morin, A. (2011). Self-awareness part 1: Definition, measures, effects, functions, and ante- 
cedents. Social and Personality Psychology Compass, 5(10), 807-823. https://doi. 
org/10.1111/j.1751-9004.2011.00387.x 

Mutweleli, S., Mundi, S., & Mwathe, G. (2024). Scenario-based assessments: Experience from 
East Africa. In E. Care, M. Giacomazzi, & J. K. Mugo (Eds.), The contextualisation of 21st 
century skills: Assessment in East Africa. Springer. 

Nakabugo, G., Madanda, B., & Kaburu, A. (2024). Opportunities and challenges in household- 
based assessment of life skills. In E. Care, M. Giacomazzi, & J. K. Mugo (Eds.), The contextu- 
alisation of 21st century skills: Assessment in East Africa. Springer. 

Organisation for Economic Co-operation and Development [OECD]. (2018). The future of educa- 
tion and skills: Education 2030. https://www.oecd.org/education/2030/E2030%20Position%20 
Paper%20(05.04.2018).pdf 

Salleh, A., Mahmud, Z., Joorabchi, T. N., Amat, S., & Hamzah, I. (2017). Measuring values in 
modern school system. International Journal of Modern Education Studies, 1(1), 28-45. 
https://doi.org/10.51383/ijonmes.2017.11 

Salovey, P., & Mayer, J. D. (1990). Emotional intelligence. Imagination, Cognition and Personality, 
9(3), 185-211. 

Seglow, J. (2016). Hate speech, dignity and self-respect. Ethic Theory Moral Practice, 19(5), 
1103-1116. https://doi.org/10.1007/s 10677-016-9744-3 

Serwanga, E., & Atuheire, G. (2021). The understanding self-awareness in Kenya, Tanzania, and 
Uganda. A report for the Assessment of Life Skills and Values in East Africa (ALiVE) project. 
RELI. https://reliafrica.org/alive/ 

Silvia, P. J., & Duval, T. S. (2001). Objective self-awareness theory: Recent progress and enduring 
problems. Personality and Social Psychology Review, 5(3), 230-241. https://doi.org/10.1207/ 
$15327957PSPR0503_4 

Sutton, A. (2016). Measuring the effects of self-awareness: Construction of the self-awareness out- 
comes questionnaire. Europe’s Journal of Psychology, 12(4), 645—658. https://doi.org/10.5964/ 
ejop.v12i4.1178 

Sutton, A., Williams, H. M., & Allinson, C. W. (2015). A longitudinal, mixed method evaluation 
of self-awareness training in the workplace. European Journal of Training and Development, 
39(7), 610-627. https://doi.org/10.1108/EJTD-04-2015-003 1 


5 Self-Awareness and Respect in East Africa: A Contextualised Approach to Defining... 79 


Svalgaard, L. (2018). The critical moment of transition: Staying with and acting on newly 
gained self- and social awareness. Management Learning, 49(3), 278-294. https://doi. 
org/10.1177/1350507617748548 

Trapnell, P. D., & Campbell, J. D. (1999). Private self-consciousness and the five-factor model 
of personality: Distinguishing rumination from reflection. Journal of Personality and Social 
Psychology, 76(2), 284-304. https://doi.org/10.1037/0022-35 14.76 

United Nations Children’s Fund. (2022, April 24). Adolescents. UNICEF Data. https://data.unicef. 
org/topic/adolescents/overview/?_gl=1%2A94vb5u%2A_ga%2AMTQ3NDQWMzkxNS4xN 
jgIMDc4NTc4%2A_ga_9T3VXTE4D3%2AMTY4NTA30DU3Ny4xLjEuMT Y4NTA30Dg 
yMi4wLjAuMA 

Vaneechoutte, M. (2000). Experience, awareness and consciousness: Suggestions for definitions as 
offered by an evolutionary approach. Foundations of Science, 5(4), 429-456. https://doi.org/1 
0.1023/A:1011371811027 

Wood, L. (2009). Duties to oneself, duties of respect to others. In T. Hill (Ed.), The Blackwell guide 
to Kant’s ethics (pp. 229-251). Wiley. https://philpapers.org/rec/WOODTO 


Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 
International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, 
adaptation, distribution and reproduction in any medium or format, as long as you give appropriate 
credit to the original author(s) and the source, provide a link to the Creative Commons license and 
indicate if changes were made. 

The images or other third party material in this chapter are included in the chapter’s Creative 
Commons license, unless indicated otherwise in a credit line to the material. If material is not 
included in the chapter’s Creative Commons license and your intended use is not permitted by 
statutory regulation or exceeds the permitted use, you will need to obtain permission directly from 
the copyright holder. 


Chapter 6 A) 
Collaboration in East Africa: gag 
A Contextualised Approach to Defining 

the Construct 


Claire Scoular © and David Alelah Otieno 


Abstract Collaboration has been highlighted internationally as a key skill for 
learning, working, and living in the twenty-first century. However, to teach it well, 
enhance its performance, and measure its growth, it is essential to have a clear and 
consistent definition of the skill. There are a number of frameworks that describe 
collaboration in a way that is meaningful to learning and growth. Despite some dif- 
ferences across frameworks, it is clear there is a common core set of contributing 
subskills. This suggests that collaboration is of global interest, and that there are 
components that transcend national or cultural specificities. Notwithstanding, defi- 
nitions and frameworks need to be suitable for the context in which they will be 
applied, ensuring the approach is integrated and sustainable in education systems. 
One focus of the ALIVE project was to develop a collaboration framework suitable 
and relevant for the sub-Saharan African context. The approach included auditing 
frameworks internationally, curricula regionally, and conducting an ethnographic 
study. The resulting framework was used in the development of assessment tasks to 
sample the skill among adolescents in Kenya, Tanzania, and Uganda. The issues 
highlighted in this chapter reveal the need for further study of collaboration in spe- 
cific contexts in order to understand local variations and ensure optimal approaches 
to measurement. 


6.1 Introduction 


For many education systems, there is increasing focus on integrating skills that are 
deemed to be relevant for learning in the twenty-first century. This increased focus 
is reflected by many international organisations which emphasise the high demand 
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for such skills in the workforce. For example, the World Bank (2019) issued a call 
for action for governments to invest in building the skills in demand in the labour 
market during schooling. One skill that is consistently mentioned across education 
systems and organisations is collaboration. The need to develop collaborative skills 
has been voiced by governments in response to the complex, multi-layered chal- 
lenges that learners and workers are likely to face in the future (UNESCO, 2015). 
Digitisation, demographic shifts, and globalisation have transformed daily social 
interactions, schooling, and the workplace, and require individuals to work collab- 
oratively to make decisions and solve problems. The OECD (2019a) has couched 
this skill as social-emotional, to emphasise the need for individuals to adapt to and 
engage with their environment and with others. These imperatives require learners 
to develop the skills to work effectively with their peers to address complex multi- 
lateral, regional, and historic challenges (OECD, 2019b). The World Bank (2010) 
has likewise emphasised the importance of developing social skills, including team- 
work and adaptability, “to collaborate with and motivate others in a team, manage 
client relations, exercise leadership, resolve conflicts, and develop social networks” 
(p. 14). For both OECD and the World Bank, collaboration can be framed as a skill 
that supports the attainment of social and economic goals across various contexts. 

Within classrooms, collaboration has been described by the UNESCO International 
Bureau of Education (2016) as involving social relationships between learners that 
foster “positive interdependence, individual accountability, and interpersonal skills” 
(p. 140). Educators are therefore framed as facilitators who skilfully develop and 
enhance their learners’ education journeys by using pedagogical strategies and tasks 
that are challenging, stimulating, and meaningful. Importantly, collaboration is dis- 
tinct from cooperation; the former is characterised by learners working in a coordi- 
nated fashion to solve a problem, while the latter involves the division of work 
responsibilities between learners (UNESCO International Bureau of Education, 2016). 

Assessment of Life Skills and Values in East Africa (ALIVE) is an initiative of 
the Regional Education and Learning Initiative (RELI), a network of more than 70 
civil society organizations in East Africa. ALiVE is a collaboration within itself 
seeking to generate evidence on life skills, including collaboration, and use this 
evidence to engage in public policy reforms as well as strengthen local capacities in 
life skills competencies in Kenya, Tanzania, and Uganda specifically. 

In seeking to respond to economic and societal challenges, curriculum reforms 
since the 1990s in East African countries (and other sub-Saharan countries) have 
generally shifted away from educator-centred and content-focused approaches, 
towards a more learner-centred approach, and emphasising thinking as well as 
social-emotional skills. While barriers that limit education access and equity, and 
traditional designs and implementation of curriculum, assessment, and pedagogy, 
continue to challenge the pace of progress in East African countries (Chisholm & 
Leyendecker, 2008; Cunningham, 2018), these reforms are resulting in improve- 
ments. For example, the process of shifting Kenya’s education sector towards a 
competency-based curriculum, as well as reforming educator professional develop- 
ment, textbooks, and local school management practices, have resulted in this coun- 
try becoming a top performer across East Africa (Simon, 2022; World Bank, 2022), 
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while Tanzania’s Free Education Policy for secondary education (Standards 1—4) 
has shown signs of improvement in learner enrolment rates and increased educator 
numbers since 2015 (Mashala, 2019). Collaboration as a skill has been identified as 
essential for learning and life in these countries and was therefore made a priority 
for defining and measuring as part of the ALIVE initiative. 


6.1.1 What Is Collaboration? 


Social interaction is critical for learning and plays a positive role in developing 
cognitive ability (Wittrock, 1989). In the context of education, collaboration occurs 
when learners work together to achieve a common goal in a shared learning envi- 
ronment (Underwood & Underwood, 1999). Generally, the nature of collaboration 
tends to focus on ability to learn from the interactive situation (O’ Neil et al., 2003). 
Social interactions that occur during collaboration such as discussing, justifying 
arguments, and negotiating with others can help to improve individual understand- 
ing (van Boxtel et al., 2000). Learners process information differently when they 
work in groups compared to working independently. 

A focus of early research on developing social skills focused on teamwork or 
cooperation, which depend upon symmetry of action with learners working on par- 
allel tasks and eventually bringing the different tasks together (Dillenbourg, 1999). 
By contrast, collaboration requires interdependent teams or individuals that rely on 
the actions of others and, in particular, on the need to share responsibility and 
engage in the active division of labour (von Davier & Halpin, 2013; Hesse et al., 
2015; Lai, 2011). 

A comprehensive understanding of what constitutes the fundamental building 
blocks of collaboration, and how it develops and changes over time, will support 
effective methods of assessing, monitoring, and teaching it. A detailed definitional 
framework of collaboration can provide the means through which to build reliable 
and valid assessment tools to measure the skill. Four conceptual structures useful 
for contextualising ALiVE’s development of an assessment framework for collabo- 
ration are discussed here. These structures demonstrate the commonalities across 
definitions, while also explaining their differences. 

As part of the Assessment and Teaching of 21st Century Skills project (Griffin & 
Care, 2015), Hesse et al. (2015) broke collaboration into three strands: Participation, 
Perspective Taking and Social Regulation. Participation refers to learners’ engage- 
ment with the task, the extent to which they persevere to solve the problem and how 
well they interact with others. Perspective taking focuses on the quality of the inter- 
action between learners during collaborative problem solving, such as how learners 
respond and adapt to one another. Social regulation refers to how learners navigate 
the collaborative space and includes negotiating and resolving differences, evaluat- 
ing their self and their peers and taking responsibility for the solving of the problem. 
This three-strand structure was built for the ‘social’ aspect of the two-dimensional 
notion of collaborative problem solving, the social and the cognitive. Its situating 
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within this broader context explains the very active nature of each of the strands, 
with each providing critical input. 

A framework that built on the work of Hesse et al. is that of Scoular et al. (2020b)— 
the Collaboration Skills Development Framework—which depicts collaboration as 
strands and subordinate aspects (Fig. 6.1). The similarity of the strands with those of 
Hesse et al. (2015) point to the commonality of understandings about the construct. 
For Scoular et al. (2020b), strands refer to overarching components, while aspects 
refer to specific content within strands. The aspects relate to specific behaviours that 
can be observed and monitored in the classroom and in real-world contexts. These are 
designed to facilitate teaching and guide assessment. For example, educators can 
write assessment items to measure specific aspects, or integrate teaching of just one 
aspect into a lesson, rather than the whole skill. This educational perspective accounts 
for the deconstructed nature of the framework—into actionable aspects. 


Collaboration 


Strand 1: Building shared 
understanding 


Strand 2: Collectively 


contributing Strand 3: Regulating 


Aspect 1.1: 
Communicates with 
others 


Aspect 1.2: Pools 
resources and 
information 


Aspect 1.3: Negotiates 
roles and responsibilities 


Aspect 2.1: Participates 
in the group 


Aspect 2.2: Recognises 
contributions of others 


Aspect 2.3: Engages 
roles and responsbilities 


Aspect 3.1: Ensures own 
contributions are 
constructive 


Aspect 3.2: Resolves 
differences 


Aspect 3.3: Maintains 
shared understanding 


Aspect 3.4: Adapts 
behaviour and 
contributions for others 


Fig. 6.1 Collaboration skill development framework. (Scoular et al., 2020b) 
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The OECD’s Assessment Framework for its Study of Social and Emotional Skills 
described collaboration as the degree to which individuals can successfully work 
together while maintaining positive relationships and minimising conflicts 
(KankaraS & Suarez-Alvarez, 2019). Inherent within this description is a “concern 
for the well-being of others” (p. 39), and an emphasis on three subskills: 


e Empathy: involves perspective taking and having concern for the well-being 
of others 

e Co-operation: harmonious living with others 

e Trust: a general assumption that others have good intentions. 


These subskills, while reflecting the perspective taking concept in other frame- 
works, highlight the social-emotional context of the OECD study. There is less 
emphasis on action, and more on the quality of interaction. 

The Cambridge Life Competencies Framework as described by the Cambridge 
Language Research Team (2020) includes collaboration as a key competency. 
Within this framework, collaboration is seen to have occurred when “learners work 
well together in groups through actively taking part in group activities, listening to 
others, sharing tasks and finding solutions to problems” (p. 3). The skill is com- 
posed of four core areas and their corresponding components (Table 6.1). Most 
strongly aligned with the OECD framework, this framework reflects the community 
and lifelong dimensions of ‘life competencies.’ It reflects the action of collaboration 
but in all respects draws attention to ‘the other.’ 

Within the collaboration frameworks mentioned, the elements that are common 
include: participation and responsibility, communication, and regulation. Table 6.2 
shows a comparison of the frameworks and includes ALiVE’s adopted structure. 


6.2 Concept of Collaboration in ALIVE 


Comparing the four frameworks supported the development of an operating 
structure for collaboration for use in ALIVE. Three dimensions of collaboration 
were defined for the ALiVE framework: working together, negotiation, and 
communication, each including associated subskills. In ALIVE, collaboration 
was viewed as an end in itself—a domain-general skill necessary for learning 
and engaging effectively in work and society. The focus is on the nature and 
quality of the collaborative interactions among group members (Fontana et al., 
2022). In exploring collaboration across Kenya, Tanzania, and Uganda, the first 
step was to conduct a rapid ethnographic study to understand local perspectives. 
Teachers, parents, community leaders and adolescents were asked to share their 
views on collaboration in terms of its meaning, and how it could be assessed. 
From the study, 187 of the participants’ described collaboration as ‘working 
together’. However, in practice the emphasis is not on working together, but on 
being together. Approximately 15% of the participants defined it as ‘teamwork’ 
or ‘cooperation’. 
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Table 6.1 Cambridge life competency framework for collaboration 


Core area 


Taking personal responsibility for own contribution to a group task. Refers to 
active participation in group activities and roles, the sharing of ideas, and a 
willingness to explain their contributions. 


Encouraging effective group interaction. References the ability to listen to 
and acknowledge others’ aligned views/conflicting views; utilise goal- 
orientated strategies within the group (e.g., taking turns, constructive 
feedback, and provide potential solutions); and establish positive and 
supportive group relationships. 


Managing the sharing of tasks in a group activity. Refers to the ability to 
identify tasks that need to be completed by the group and appropriately 
managing the distribution of task responsibilities between group members. 


Core area 
component 


Actively 
contributing to a 
task 

Taking on 
different roles 
Listening and 
responding 
respectfully 
Establishing 
ways of working 
together 
Engaging and 
supporting others 
Agreeing what 
needs to be done 
Managing the 
distribution of 
tasks 


Working towards task completion. Ensure that progress is being made to 
achieve the group’s task goals. This could include engaging in evaluation 
processes, identifying the most appropriate solution to complete the task, and 
engaging in conflict resolution. 


Ensuring 
progress towards 
a goal 
Identifying issues 
and challenges 
Resolving issues 


Table 6.2 Identification of common elements across frameworks 


Participating and 
Frameworks responsibility Communication Regulation 
ATC21S (Hesse et al., Participation Interaction Social regulation; 
2015) perspective taking 
Scoular et al. (2020b) Collective contribution | Build shared Regulation 
understanding 

“OECD (2019a, b) Co-operation Empathy Trust 
Cambridge Language Responsibility; toward | Group interaction Sharing of tasks 
Research Team (2020) task completion 

“ALIVE (Fontana et al., | Working together Communication Negotiation 
2022) 


From the ethnographic study, very few participants said that collaboration meant 
achieving a common goal, which element was a key feature in all of the frameworks 
reviewed here. Several of the participants elaborated that achieving a shared goal is 
not necessarily what drives collaboration, but rather it is the element of doing some- 
thing together. The terms sharing, unity, and love were frequently used by the par- 
ticipants to define collaboration. Further, a crucial component of collaboration 
identified by participants is belonging to the community. 
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COLLABORATION 


COMMUNICATION WORKING TOGETHER NEGOTIATION 


Receptive Planning Perspective taking 


Responsibility Consensus building 


Participation 


Fig. 6.2 The ALiVE collaboration framework 


This participant feedback was used to refine the definition, while also drawing on 
existing frameworks and literature, and on the national education curricula in the 
three countries. Collaboration between countries on this step was not only in the 
spirit of the ALIVE endeavour but was integral to ensure a framework relevant 
within each country. The ALiVE framework depicts collaboration across three 
dimensions and their subskills (Fig. 6.2). 


6.2.1 Dimension 1: Communication 


Communication is the act of transferring information from one person to another, 
whether vocally, visually, or non-verbally. Communication is the essential channel 
used by collaborative partners to send and receive information, keep one another 
informed, and convey opinions to influence group actions (Mattessich et al., 2001). 
Collaborators must explicitly communicate their ideas, knowledge, and opinions 
when working on a task in their groups, as it is through this communication that 
they can guide the direction and process when working on a task (Driskell 
et al., 2018). 
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This dimension targets an individual’s ability to listen (receptive) and speak 
(expressive). Receptive communication is the ability to listen and understand words 
and language. Much of this type of language is receiving and interpreting commu- 
nication from our environment, experiences, and of course others. Expressive com- 
munication on the other hand is the ability to speak or create narrative. It involves 
the use of words, sentences, gestures, and writing to create a message or convey 
meaning. Both receptive and expressive communication is essential for effective 
collaboration (Lai, 2011). When participants explain their ideas to others their 
understanding and thinking becomes explicit and self-awareness and understanding 
is likely to increase (Scoular & Care, 2019). Providing clear explanations enhances 
collaborative interactions. This is critical to the act of working towards a com- 
mon goal. 


6.2.2 Dimension 2: Working Together 


Collaboration goes beyond coordinating and cooperating; it requires some interde- 
pendency between participants. Collaboration involves participants working 
together on the same task, rather than a division of labour working in parallel on 
separate elements of a task (Lai, 2011). This interdependency should commence 
from the planning stage of a joint initiative. In collaboration, setting goals is a pre- 
requisite to driving plans into execution. Goal specificity improves a group’s perfor- 
mance (Weldon & Weingart, 1993) and goals need to be appropriate to the task to 
be efficient (Stevens & Campion, 1994). PISA’s CPS framework identifies goal set- 
ting as a collaborative process and specifies that students should be able to identify 
and describe the tasks that need to be completed, which specifically includes creat- 
ing team goals (OECD, 2013). 

Planning requires taking responsibility to solve a problem. Proficient collab- 
orative problem solvers should encourage shared responsibility for the task. 
Some teamwork models align learner ability to take responsibility with their 
ability to lead (O’ Neil et al., 2003), although collaboration does not require one 
individual to take sole responsibility or leadership for the task—there must be a 
distribution of responsibility. Hesse et al. (2015) suggest that if learners do not 
adopt shared responsibility, group members may disengage from a task. The 
group needs to negotiate their responsibilities to ensure that there is the best 
match to the expertise, information, or skills held by any given group member 
(Scoular, 2019). 

Finally, proficient collaborators will participate throughout the activity and 
see it through to the end goal or solution (DiCerbo, 2014). Proficient collabora- 
tors will make multiple attempts at group tasks and try alternative strategies to 
reach the end goal even during difficult situations or problems (Scoular & 
Care, 2019). 
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6.2.3 Dimension 3: Negotiation 


When working collaboratively, learners need to find effective ways of resolving any 
conflicts that arise when trying to reach the common goal. Learners may bring dif- 
ferent opinions but these must be with careful consideration of the views of others 
(Scoular, 2019). Negotiation skills are the result of rational thinking based on 
informed choices and interacting such that ideas are both shared and accepted across 
the collaborators. 

Perspective taking is essential to negotiating in collaborative teams. Understanding 
the perspective of others can change interactions and how individuals respond and 
adapt to one another (Dehler et al., 2007). For example, the ATC21S project assessed 
students’ audience awareness during online collaborative tasks and observed that 
students with higher proficiency demonstrated fewer activities before stopping and 
waiting for their partner to provide their perspective (Care et al., 2015). The ability 
to adapt one’s interaction style for others is based upon individual awareness of oth- 
ers’ ability, style and needs (Hesse et al., 2015). Proficient collaborators acknowl- 
edge that others may have a different perspective, which may be beneficial to the 
group. Listening to, acknowledging, understanding, and critiquing the perspective 
of others can lead to changes in a person’s behaviour and group dynamics. 


6.2.4 Measuring and Monitoring Collaboration 


Each of the frameworks discussed in this chapter adopts the assumption that col- 
laboration skills can grow, can be improved through teaching and intervention, and 
can be measured and monitored. This view is signified by the developmental frame- 
work of each with levels of proficiency outlined from early to more advanced appli- 
cation. A key feature of these levels is that they are not linked to specific years of 
schooling. Learners of the same age can be at very different points in their skill 
development. Assessment of functioning levels are an alternative to judging success 
only in terms of year-level or curriculum-based achievement standards. This 
approach is akin to that adopted in ALIVE. 

Developmental frameworks not only identify increasing levels of proficiency but 
may also link these with maturational features from infancy to adulthood. For exam- 
ple, the OECD’s collaboration framework provides initial evidence that might be 
used to support the development of a skills progression (KankaraS & Suarez- 
Alvarez, 2019). In this framework, collaboration has been framed using the sub- 
skills empathy, co-operation, and trust, with the presence, absence, and development 
of these subskills having the potential to indicate levels of collaboration. The pres- 
ence of empathy during childhood, for example, has been associated with 6 and 
9 year old children displaying social justice values (e.g., fair treatment and equality) 
later in life at age 12, while the absence or lack of empathy during early childhood 
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has been associated with adverse outcomes during the adolescent years. Observable 
behaviours regarding empathy might include whether a child consoles another or 
sympathises with the poor, while a lack of empathy might be exemplified by a ten- 
dency to misinterpret, ignore, or disregard what a person feels. Cooperation has also 
been observed to trend in predictable ways. Cooperation or cooperative behaviours 
typically decline during later childhood (e.g., between 6 and 10 years old) and 
increase again as adolescents approach their twenties; suggesting that some aspects 
of cooperation can be shaped by developmental, epistemological, or environmental 
factors. Children who are capable of cooperating easily make friends and respect 
group decisions, while those who are less capable may be more prone to arguments 
and conflicts or be unable to compromise (Kankara$ & Suarez-Alvarez, 2019). 
Finally, individuals who are more trusting are more likely to lend their belongings 
and avoid conflict, while those who are less trusting are prone to arguments with 
others and being unable to compromise. 

In the case of the ALIVE collaboration framework, this was developed specific 
to understandings of likely skills demonstrations by adolescents. Hence identifica- 
tion of functioning levels in Table 6.3 represents a relatively narrow ‘developmen- 
tal’ range for the skill. Notwithstanding, the range as described can be used as a 
skeleton upon which more flesh can grow as the framework is used to develop 
assessment tasks targeted at a wider age range. 


Table 6.3 Qualitative descriptors of collaboration levels of proficiency 


| Level 1 | Level 2 Level 3 Level 4 
Collaboration | Does not Is attentive to the | Collaborates through | Collaborates 
overall engage either | discussion and speaking and being | through taking 
by being may query the attentive in positions and 
attentive to views of others, | discussions, and contributing ideas, 
discussion, but does not engaging actively in | prompting others, 
speaking, or contribute in performance tasks and being attentive 
through action | word or action to others’ inputs 
Communication | Does not Appears to listen | Communicates by Engages with 
appear to be to the discussion | listening and others through 
attentive to the speaking listening, 
activity speaking, and 
prompting others 
to contribute 
Working Does not Engages slightly | Contributes to the Engages in the 
together engage inthe |in the activity as | task actively activity oneself as 
activity or with | a follower well as involving 
others others 
Negotiation Does not Participates Contributes through | Engages through 
appear to be through querying | questioning others | questioning and 
attentive to the | the inputs of and providing own | responding to 
activity others perspective others and 
contributing own 
ideas 
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The assessment of collaboration in ALiVE was based on performance-based 
tasks. The situations presented to adolescents were based on real life contexts— 
making use of day-to-day situations that would be familiar to respondents. 
Participants were presented with a task followed by a series of questions that 
required the adolescents to focus on how they would work together towards a com- 
mon goal. The tasks were designed to target a range of sub-skills in the collabora- 
tion framework. The observable behaviours of the adolescents can be interpreted 
within the proficiency levels outlined in Table 6.3. The levels indicate progressive 
sophistication and complexity over the levels. 


6.2.5 Assessment Considerations 


Collaboration is a complex skill, with many subskills and layers. Design of assess- 
ments to capture collaborative proficiency is complex but requires simplicity in the 
actual approach. A task must be sufficiently ‘dimensional’ that it requires varied 
inputs from several individuals, and sufficiently simple to enable the capture of 
those individuals’ contributions. The form of tasks and mode of assessment delivery 
is therefore seminal to useful assessment. In the case of ALIVE, the mode of assess- 
ment delivery was at household level. In other words, adolescents from the com- 
munity would be assessed in their home environments, and these adolescents would 
be assessed doing familiar tasks that would not require additional information 
inputs. All of these elements combined to guide the process of task development. 
Three such elements are outlined below, and how these elements were managed is 
described through discussion of one task developed for the ALiVE large scale 
assessment program. 

The multi-dimensional nature of a complex skill like collaboration requires ana- 
lytical decomposition of the skill in order to ensure that what is being measured is 
true to the construct. In this case, the assessment tasks must be designed to require 
collaboration, rather than teamwork or cooperation, and to elicit behaviours from 
which we can judge the capacity to collaborate. Each of these characteristics has 
implications for design of the tasks. One such consideration is the distribution of 
resources. The nature of collaboration relies upon asymmetry of resources between 
individuals, representing a real-world view of diversity of expertise, knowledge, 
and information (Care et al., 2016). Collaboration requires informational or compe- 
tence diversity to ensure there is a purpose in working together, rather than working 
alone (Scoular & Care, 2019; De Wit & Greer, 2008). Therefore, assessments in 
which individuals each possess unique resources that are required for task comple- 
tion, provides for more authentic coverage of the construct. If assessment tasks are 
presented that require individuals to work together, and if there is a mutual benefit 
from working together, then they are more likely to stay engaged in the task and see 
the benefit of contributions from others for themselves. At most basic therefore, 
tasks must be designed such that individual contributions to the goal can be observed. 
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The measurement of a complex skill such as collaboration is not a matter of iden- 
tifying success or failure. Interest in measurement of life skills tends to centre on 
degree of skill rather than presence or absence. Assessing for degrees of difference in 
proficiency is a challenging endeavour requiring innovative approaches. At most 
basic, tasks must provide the facility to stimulate varying levels of proficiency from 
adolescents, providing responses that can be evaluated using standardised guidelines. 

Another consideration is group composition. The issue of how groups should be 
formed is an important one, since individuals may perform differently depending on 
the group to which they are assigned, with factors such as differences in ability 
(Wildman et al., 2012), personality characteristics (McGivney et al., 2008), and 
gender (Bear & Wooley, 2011) all potentially influencing how individuals within a 
group might collaborate. Decisions about group composition for any assessment of 
collaboration should be made with the purpose and the nature of the evidence it 
aims to collect firmly in mind, with the knowledge that such decisions may influ- 
ence what aspects of the skill can be elicited by the assessment. 


6.2.6 A Collaborative Task 


The design of tasks to capture adolescents’ collaborative proficiencies responded to 
the practicalities of ALiVE’s large scale household-based assessment program. 
How the considerations identified above were factored into the task design is 
described with reference to one of the tasks used—the making of a ball. The initial 
instructions given to each group of four adolescents were: “As a group, discuss and 
agree on available materials that can be used in making a ball, and then make a ball.” 
Additional prompts were given at each of three stages in task completion: (1) dis- 
cussion of possible materials to use; (2) agreement on the materials; and (3) the 
making of the ball. The stepped instructions were necessary in order to partition the 
different activities in such a way that the subskills (or dimensions) of collaboration 
could be observed. The initial discussion provided data for the observing Test 
Administrators to rate communication; the agreement process provided data for rat- 
ing of negotiation skills; and the final activity provided data on how the adolescents 
worked together. Without this explicit partitioning, it would not have been possible 
for the Test Administrators to capture the behaviours and evaluate these against the 
rubrics designed to rate them. 

In turn, each activity stage and its related subskill provide the opportunity for 
young people to demonstrate their varied proficiencies. Scoring rubrics are shown 
in the context of each of the subskills in Table 6.4. As can be seen, each of the crite- 
ria across levels 0 to 3 captures higher levels of competency than the one prior. 
These criteria capture relatively broad levels of proficiency, aligned with the sim- 
plicity of the task design, and they fulfil ALIVE’s reporting needs. Other assessment 
environments, particularly those supported by online technologies, can capture 
more finely delineated levels of proficiency (e.g., Griffin et al., 2013) as well as 
more of the behaviours that contribute to collaboration (Scoular et al., 2020a). 
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Table 6.4 Activities, structure, subskills and scoring criteria for ‘make a ball’ task 


Performance Criteria to score proficiency levels 
Activity | Subskill indicator | 0 1 2 3 
Discuss | Communication | Ability to Does not | Does not | Speaks and | Speaks, 
materials speak and speak and | speak but | is attentive | prompts 
to be listen is not is attentive | in the others and 
used in attentive in | in the discussion | is attentive 
making the discussion in the 
the ball discussion discussion 
Agree on | Negotiation Ability to Does not | Questions | Questions | Questions 
materials express own | question the views | the views | the views of 
to be opinion and | the views |ofothers |ofothers | others, takes 
used in ability to of others but does and takes a | a position 
making accept others | andis not | nottakea_ | position and adds 
the ball opinion attentive in | position something 
the new, 
discussion contributes 
ideas 
Make the | Working Ability to Does not | Either Collects Collects 
ball together participate collect collects materials | materials, 
(in making materials | materials | and involves 
the ball) and does or engages | engages in | others and 
not engage |in making | making the | makes the 
in making | the ball ball alone | ball 
the ball 


A feature of the collaboration assessment in ALIVE was its necessary simplicity. 
Given reliance on daily life to act as the host and the stimulus for collaboration, and 
on the household-based administration mode, the provision of different resources to 
individuals in each of the adolescent groups was not viable. Therefore, tasks needed 
to be designed such that they required input from more than one individual in a less 
defined manner than has been the case in technology-supported large-scale pro- 
grams such as OECD’s PISA and ATC21S. In this way, although the individuals 
within a group did not have access to different resources to bring to bear in the task, 
thus fulfilling the asymmetry of tasks criterion, they were required to work together 
in order for the goal of the task to be achieved. 

The final element to recognise in the assessment of collaboration in ALIVE is 
group composition and its dynamic. This concern could be controlled for only by 
virtue of gender identity of the participating adolescents. The nature of the collabora- 
tion tasks was such that adolescents, regardless of gender, could easily engage in the 
activity. In order to control for possible gender effects, the group composition was two 
males and two females. The results (Ariapa et al., 2024; Chap. 10, this volume) of the 
large-scale assessment shows negligible differences in performance across gender, but 
since there were no single sex groups, the possible influence of gender could not be 
ascertained. A second variable in group composition was age and education level of 
the four individuals. Due to the sampling design within the local communities, it was 
not possible to control for this. It is therefore plausible that age and education 
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differences could have influenced results, with presumably older adolescents taking 
stronger roles in the activities. The results show that older adolescents in fact did show 
greater proficiencies in collaboration, and the degree to which this might be due to 
natural increase in proficiency with age, versus a dominant effect on younger collabo- 
rators, could not be ascertained. 

To measure collaboration effectively then, there is a clear set of criteria for task 
requirements and development. Tasks need to be sufficiently innovative in capturing 
the multi-dimensionality of the skills. Asymmetrical tasks in which learners have 
access to different resources, and opportunities to work through a process rather 
than just respond to stimuli, are both essential for capturing varying levels of profi- 
ciency. The subskills in the definitional framework provide an easy transition from 
task to rubric criteria used to judge performance. In other words, the rubrics and 
associated criteria neatly map back to the framework to ensure a range of subskills 
are being captured, and in a way that is described as per the definition. Each of these 
elements has its own contexts and considerations for both development and what 
would be anticipated of learners. For example, decisions about group composition 
need to be considered with the context and the demands of the task in mind. It is 
possible that composition of groups could have an impact at the individual level 
depending on inclusion or exclusion of specific others, or on demographic criteria, 
and should therefore be considered at the design stage. 


6.3 Conclusion 


Collaboration is a vital component of life, whether it is in the workplace, the home, 
society, or the classroom. It is one of the fundamental skills that was prioritised 
across the three countries (Kenya, Tanzania, and Uganda) in the ALIVE project. 
There are some difficulties in measurement and evaluation of collaboration gener- 
ally, and in integrating the skill into educational systems. Throughout this chapter, 
the need to define and describe collaboration is key to understanding and measuring 
the skill. While there are differing frameworks describing collaboration, it is clear 
that there is a set of underpinning skills across these frameworks internationally. 
This suggests that collaboration is an important skill of global interest, and there are 
components that transcend national or cultural specificities. 

The chapter also outlines that beyond the importance of defining the skill of col- 
laboration, developing assessment tasks that provide the opportunity to measure 
ability and monitor progress is essential. The ALIVE project has made progress in 
identifying task development considerations in the context of East African coun- 
tries, focusing on measuring collaboration in adolescents using performance-based 
tasks at the household level. Clearly collaboration is a complex skill to measure 
given that its application relies on the involvement of others and requires tasks suf- 
ficiently complex and multi-faceted to enable learners to demonstrate it. Capacity 
development in assessment processes needs to take place among educators at all 
education levels and is a crucial step in ensuring the approach is integrated and 
sustainable in education systems. 
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Lastly, the chapter highlights some of the gaps in research at present, for 
example examining the impact of other variables of interest, such as gender and 
age, on collaboration. The issues highlighted in the chapter reveal the need for 
continued exploration of collaboration in specific contexts in order to under- 
stand the contributing dimensions and subskills to local demonstrations of 
the skill. 
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Abstract To address the challenges originating from changes in the global market 
and with technological progress, sub-Saharan Africa (SSA) is adopting more holis- 
tic education systems that offer lifelong competence for workers in the twenty-first 
century. This transition requires integration of complex cognitive and social/inter- 
personal competencies, such as critical thinking, teamwork, cultural and diversity 
awareness, multilingualism, and the use of digital technologies into the traditional 
educational curricula. However, the transition from traditional to holistic curricula 
is complicated. Issues include how twenty-first century skills are defined in SSA, 
how they can be taught, how they can be integrated into curricula, and how they can 
be assessed. A review of the literature on assessments was first conducted in order 
to review approaches and tools used to measure twenty-first century skills in 
SSA. Five assessment approaches were identified: scenario-based, questionnaire, 
video recording and direct observation, portfolio, and technology-based. Seven 
tools that met study criteria were examined along five dimensions: purpose, type or 
form, target population, context, and specific skills, in order to determine their util- 
ity for assessment of twenty-first century skills. Findings indicate that five of the 
seven assessment tools support summative purposes while two support formative 
assessment. Further to this, two tools were designed for large-scale assessment and 
three targeted adolescents. In terms of method, scenario-based and self-report were 
the most common approaches used to collect information on twenty-first century 
skills in SSA. Notably, the outcomes of scenario-based assessments provided com- 
pelling evidence of proficiencies, demonstrating the method’s efficiency in task cre- 
ation, analysis, and scoring rubrics that provide clear distinctions across 
performance levels. 
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7.1 Introduction 


Emerging global market trends and technological progress have led to rapid changes 
as well as job ambiguity in the workplace. Consequently, learning that would enable 
workers to acquire job opportunities in the twenty-first century has shifted from 
focusing only on traditional cognitive skills and discipline-specific knowledge to 
the integration of complex cognitive and social/interpersonal competencies, also 
referred to as twenty-first century skills, such as critical thinking, teamwork, cul- 
tural and diversity awareness, multilingualism, and use of digital technologies 
(Joynes et al., 2019). There continues to be a large discrepancy between what young 
people are learning in school and the types of skills that people need to succeed in 
the workforce and today’s society. 

Over the last two decades it has been acknowledged that existing education cur- 
ricula are no longer adequate drivers for success in the increasingly competitive 
environment (Voogt & Roblin, 2012). To address this problem, international organ- 
isations such as the United Nations Educational, Scientific and Cultural Organization 
(UNESCO), Partnership for Twenty-first Century Skills (P21; Battelle for Kids, 
2019) and Assessment and Teaching of Twenty-first Century Skills (ATC21S; Care 
et al., 2018), have developed initiatives to place these skills at the centre of learning 
(Gonzélez-Salamanca et al., 2020). Furthermore, the United Nations 2030 
Sustainable Development Goal 4 promotes the need for learners to obtain more 
comprehensive skills beyond traditional numeracy and literacy, including global 
citizenship skills, as well as the skills needed to promote sustainable development. 

In agreement with these initiatives and Sustainable Development Goal 4, SSA is 
adopting more holistic education systems that offer lifelong competence for the 
twenty-first century, and at the same time is embracing innovation, cultural and 
diversity awareness, and the use of information technology in both formal and infor- 
mal learning contexts. Nonetheless, there have been challenges, specifically around 
how to define these skills in the context of SSA, how to teach these skills in class- 
rooms and schools, how to successfully integrate the skills into curricula, and 
importantly, how to assess these skills (Kim & Care, 2020). One of the most critical 
concerns has to do with national assessment frameworks. As is also the case of 
many ‘Global North’ countries, the national assessment frameworks in SSA primar- 
ily focus on the measurement of traditional curriculum-based learning outcomes 
(Siarova et al., 2017). This has created a need to adjust and expand assessment 
frameworks in order to identify different methods of capturing the holistic nature of 
lifelong competencies suitable for twenty-first century living. 

There are four goals of this chapter: (1) to discuss some complexities of assess- 
ing twenty-first century skills; (2) to identify common methods used to assess these 
skills and discuss strengths and weaknesses of the methods; (3) to describe general 
findings from a review of existing assessments of twenty-first century skills in SSA; 
and (4) to discuss implications of the findings in the context of SSA. 
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7.2 The Complexities of Assessing Twenty-First 
Century Skills 


Societal demands of education in the twenty-first century have seen a fundamental 
shift from education in the twentieth century, where knowledge accumulation was 
highly valued. In today’s rapidly changing, globally interconnected world, it is not 
enough to memorize mathematical facts and vocabulary; instead, there is a need and 
a real demand for education systems to provide learning opportunities and experi- 
ences that align with what learners will face in the real world—where they must not 
only have information but also know how to use that information in different ways 
and in different situations. Unfortunately, there continues to be a substantial mis- 
match between the goals of some education systems—the desire to teach and learn 
twenty-first century skills—and the implementation of these goals (Care et al., 
2019a, b). 

Part of the issue is that current teaching approaches are based on offering the 
same curriculum to every learner at the same grade level. Linked with this approach 
has been the use of very similar assessment approaches, typically summative, to 
measure learning outcomes. However, the transversal skills that fall under the 
twenty-first century competencies skills umbrella are dynamic, and how they are 
exhibited varies according to the situation and context. The implication is that the 
traditional, typical way of collecting information to infer learner abilities such as for 
literacy and numeracy—paper-and-pencil, multiple-choice, short answer questions 
that ask learners to define a term or provide information about a phenomenon—can- 
not adequately capture individual abilities in these twenty-first century competen- 
cies. Although traditional assessment methods may be able to identify what 
individuals know, they are unable to demonstrate whether individuals can take what 
they know and apply that knowledge in real-life situations, where they must simul- 
taneously know which skills to use, when to use them, how to use them, and quickly 
pivot if the situation changes. Therefore, the assessment of twenty-first century 
skills must go beyond the traditional summative-type tests or examinations that 
determine whether a learner has grasped (i.e., memorized or comprehended) the 
domain of interest (Care & Kim, 2018; Galloway et al., 2017; Kim & Care, 2020; 
OECD, 2017), to the assessment of complex learning processes which are typically 
not visible, using qualitative behavioural indicators. For example, a question like 
“What is 24 x 4?” has a correct response, and it is easy to identify whether a learner 
knows the answer or can work it out. However, open-ended questions such as “What 
is the best way to approach this problem and why?” are more difficult to assess 
using traditional assessment methods since there are many ways to respond to this 
question. In addition to capturing the skills and processes underlying the competen- 
cies, assessments need to capture whether and how individuals apply these skills in 
real-life situations (Kim & Care, 2017). 

The challenge is how best to capture these behaviours. Assessments need to be 
sufficiently broad and dynamic to assess skills that are demonstrated across differ- 
ent situations and in response to different contexts, recognizing that these skills may 
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be exhibited differently depending on the situation and the learner. At the same 
time, the assessments and how data are collected using these assessments, need to 
be systematic, reliable, and practical (Vista et al., 2018). The use of technology- 
based methods has been one way to address this challenge by transforming qualita- 
tive measures into quantitative data that are scored in a consistent way (Sibberns, 
2020). Accordingly, computer-based assessment has been recognized as adding 
value to traditional paper and pencil assessment approaches. Technology-based 
assessment has the capacity to track a wide range of competencies such as thinking 
processes and interactions through which the learners arrive at their responses 
(Sibberns, 2020). Using this type of assessment seems promising; however, 
technology-based assessment does not of itself ensure validity, and access to tech- 
nology is still far from ubiquitous in SSA. This, of course, precludes access to some 
of these digitalized assessment innovations (Scardamalia et al., 2012). One thing is 
clear—the complexity of twenty-first century competencies makes it challenging to 
assess them. 

A practical challenge in assessing twenty-first century skills is that specific 
grade- or age-level expectations of learners have not been made explicit. Nor are 
there widely available curricula that outline what the learning goals are when it 
comes to these competencies. Several studies found that despite movement in pol- 
icy, teaching and assessment of these skills remains a challenge both in SSA 
(Netherlands National Commission for UNESCO, 2015; Kim & Care, 2020) and 
elsewhere (e.g., Care et al., 2019a, b; Vista et al., 2018). 


7.3 Common Methods of Assessing Twenty-First Century 
Skills in Sub-Saharan Africa 


From a review of literature specific to SSA, five approaches to measuring twenty- 
first century skills are considered here. These include scenario-based assessment, 
questionnaires, video recording and direct observations of activities, portfolio 
assessment, and technology-based assessment. Each of these methods is described 
briefly with reference to these twenty-first century skills. 


7.3.1 Scenario-Based Assessment 


Scenario-based assessment consists of a description of a situation followed by items 
(or questions) to which response is required. In the case of twenty-first century 
skills, this approach enables the task designer to ground assessment activities in the 
concrete details of a situation so as to stimulate the respondent’s multi-dimensional 
action or thinking. If scenarios are linked to episodes that are familiar to a respon- 
dent, the latter generally find it easy to engage in the task. In a study conducted 
among university students, Haynes et al. (2009) identified some benefits of the 
approach: 
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Scenarios provided all students a chance to participate and give ideas without having to 
worry about getting a bad grade for the wrong answer. Scenario assessment can even be 
more beneficial if the context in which the scenarios are applied involves working in teams. 
(Haynes et al., 2009, p. 4) 


Scenario-based assessments that are based on familiar situations remove stressors 
that might be associated with classroom-based knowledge, and so liberate those 
being tested to provide more authentic responses. In addition, the approach provides 
a good milieu within which individuals can express and explore their social, cogni- 
tive, and behavioural responses associated with target skills more broadly. Daily life 
scenarios may be perceived as permitting more nuance in responding than the cor- 
rect/incorrect response mode that is frequently associated with assessment in the 
formal education sector. 

Scenario-based assessment is subject to complexity in scoring. The development 
of scoring rubrics is complicated. If done well, rubrics can provide reliable scoring 
of quite complex behaviours. However, if rubrics are poorly expressed or not thor- 
oughly reviewed for clear thinking, the assessors are blocked from accurately repre- 
senting the proficiencies of the respondents. The process of designing and 
redesigning scenarios in response to feedback from the field is also time-consuming. 
However, if the process is handled appropriately, it becomes possible for the asses- 
sor to distinguish a poor performer from a better performer. Finally, the assessor’s 
judgment can be influenced by contrast bias during the scoring of individuals’ 
responses. This can occur when an assessor inappropriately compares a single indi- 
vidual to a group of respondents who performed extremely well or extremely poorly. 
For example, an average individual may be scored as weak if preceded by others 
who have performed extremely well on the same task. Note that contrast bias can be 
reduced if two or more assessors are involved in the assessment and then aggregate 
their scores (Yeates et al., 2015). 


7.3.2 Self-Report Questionnaires 


Self-report questionnaires are commonly used to measure participants’ attitudes 
and opinions. Items on self-report measures typically ask respondents to rate them- 
selves on a variety of factors using responses along a Likert scale, such as “strongly 
agree” to “strongly disagree”. There are several popular self-reports and/or ques- 
tionnaires that have been developed to capture a variety of skills desired in the 
twenty-first century. These include The Student Experience 21! by Battelle for Kids, 
an online suite of tools that includes a 24-item student perception survey that focuses 
on hope, engagement, belonging, and twenty-first century learning; and the PISA 
2018 assessment of Global Competence (OECD, 2017), which included a self- 
report component through which students report evaluation of own attitudes and 
competencies. 


'TheStudentExperience21 | Battelle for Kids 
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Many more surveys of twenty-first century skills continue to be developed for 
different age groups, contexts, locations, and purposes. For example, Boyaci and 
Atalay (2016) developed a 39-item survey, Twenty-first Century Learning and 
Innovation Skills, for Turkish primary students using a Likert scale to capture self- 
report responses on creativity and innovation, critical thinking problem-solving, 
and cooperation and communication. Kelley et al. (2019) developed an instrument 
for high school students to assess their self-reported proficiencies for communica- 
tion, collaboration, critical thinking, and creativity within project-based learning 
activities. Example items include: “I am confident in my ability to help the team 
solve problems and manage conflicts” (collaboration); “I am confident in my ability 
to elaborate and improve on ideas (creativity). 

Questionnaires and self-reports are often used to measure twenty-first century 
skills because they are easy to design and administer, can be completed quickly, 
tend to be less costly than technology-based assessments and open response for- 
mats, and can be easily analyzed quantitatively (Boyaci & Atalay, 2016). At the 
same time, the method is well-known and therefore familiar to many respondents, 
making it easier to use. Another benefit is the facility to compare multiple perspec- 
tives, such as the student versus the teacher (Kelley et al., 2019). 

However, the use of the Likert response method in combination with self-ratings 
has been criticized for failing to provide evidence of construct validity (Chakrabartty, 
2014). Lombardi et al. (2018) pointed out that students may lack objectivity or may 
bias their responses due to perceived social pressure. In addition, responses to hypo- 
thetical statements may not parallel how students would respond when encounter- 
ing a situation in real life (Soland et al., 2013). This is a particular issue when 
measuring twenty-first century skills that are less well-defined or more nuanced, 
and which, therefore, lack key indicators against which a respondent can evaluate 
themselves. These weaknesses are threats to validity and may pose serious concerns 
for capturing students’ abilities. 


7.3.3 Videotaping and Direct Observations 


Direct observation or video recording of individuals engaging in tasks is an assess- 
ment method for measuring practical or socio-emotional skills in which the target 
individual or group is observed in given situations to determine how they perform. 
A set of instructions is provided to the target individuals or groups about the task 
requirements, the assessment criteria, and the duration of the assessment. It may be 
that only the administrator evaluates, or that both the administrator and peers par- 
ticipate in the evaluation. A scoring guide, or set of rubrics, is used to rate the per- 
formance (Mclean & Connor, 2018; Kemp, 2001). 

A study of direct observation of social worker students in field practice revealed 
that the approach yielded more valid results when clear goals of the assessment are 
agreed upon by both the assessors and students (Irwin, 2014). Goal setting is there- 
fore viewed as an important step when preparing for direct observation assessment. 
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Goals provide a clear guide concerning what is expected of the students and may 
help in reducing their anxiety during the process. The approach has been com- 
mended for embracing greater external or contextual validity than self-rating scales, 
as behaviour is measured in the form of its natural occurrence (Nock & Kurtz, 
2005). Unlike many assessment approaches that focus on relatively narrow factors 
such as frequency, direct observation provides the opportunity to gather a wide 
range of contextual information about the events that occur during task perfor- 
mance, including the antecedents and consequences of observed behaviour (Nock & 
Prinstein, 2004). 

Despite these benefits, a number of issues have been identified. First, the direct 
observation approach is costly in terms of administration time and money, while 
video-based approaches also require adequate technologies and equipment (Irwin, 
2014). Second, direct observation may induce respondent-assessor reactivity due to 
the obtrusiveness of the assessor as the observer. In addition, perceptual bias may 
influence the assessor’s interpretation of behaviours that in turn is used to infer good 
or poor quality (Skinner et al., 2000). Third, in the case of the education context, 
there is potential for teachers to experience role conflict when they combine the role 
of an assessor who is judging students’ achievement with that of teacher who is 
responsible for developing students’ skills. Such conflict might ensue from the 
teachers’ natural tendency to help and guide a student; or from their own concerns 
that student performance might reflect on perceptions of teacher success (Kemp, 
2001). To achieve success in utilizing direct observation assessment, it is necessary 
to train the assessors and equip them with the principles of managing these issues. 

The method itself provides a clear opportunity for rich demonstration of complex 
skills, due to its open-ended nature. In order for assessment of twenty-first century 
skills to benefit from the method, the target skills must be comprehensively de- 
constructed and identified in scoring criteria, analysts must be comprehensively 
trained in observation, and sufficient resources in terms of number of observer ana- 
lysts must be available. 


7.3.4 Portfolio Assessment 


In recent years, portfolios have become a popular resource for assessing some 
twenty-first century skills. A portfolio is a collection of student work that is exam- 
ined and scored based on a set of predetermined criteria. A portfolio includes learn- 
ers’ annotated evidence, such as peer or teacher ratings, research reports, products, 
and argumentative essays, that reflect what the learner has achieved in learning 
(Davis et al., 2001). 

Portfolio assessment has the potential to assess complex phenomena such as 
attitudes and ethics that are difficult to measure using traditional classroom assess- 
ment approaches. The use of portfolio assessment in assessing competencies has the 
added advantage of showing changes in behaviour over time since it is grounded in 
self-reflection (Bialik et al., 2016; Baki & Bargin, 2004). Portfolio assessment is 
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widely used in secondary and tertiary education where ‘products’ are seen as the 
most informative sources of competence. Predominantly, portfolios have been used 
in the arts and business fields to assess individuals’ competencies. This is because it 
s a product of the knowledge and skills contribution that is being assessed, as dis- 
tinct from assessment of dynamic display of skills. 

However, one of the major concerns in using portfolio assessment is the issue of 
validity. The portfolio of work may not include a representative sample of the infor- 
mation across the targeted learning outcomes (Barton & Collins, 1997). Assessors 
may also find it difficult to verify whether the evidence is attributable to the student 
or whether it is faked or plagiarized (Davis et al., 2001). In other words, the link 
between the individual being assessed and their product is not always verifiable. 

Another challenge with portfolio assessments is that similar to open response 
measures, it can be difficult to score consistently given the myriad of evidence that 
might be part of the portfolio of work. There have been efforts to ensure that portfo- 
lios can be scored in a way that ensures high levels of reliability and validity, for 
example, by developing clearly written rubrics. In other words, assessment develop- 
ers are focused on ensuring that the scoring process is highly structured to make 
sure there is consistency in the criteria for scoring as well as increased agreement 
among raters. Another concern is assessor bias, which makes it difficult to ensure 
objectivity in scoring. One solution for this is to standardize rubrics, although again 
excessive standardization of scoring rubrics as an effort to reduce assessors’ subjec- 
tive judgment may remove the nuanced information that might be critical to captur- 
ing the target skills. 


7.3.5 Technology Based Assessment 


Technology-based assessment resources for twenty-first century skills are less 
accessible in regions that do not have high levels of technological and communica- 
tions infrastructure throughout governance, societal, and education systems. 
Technology-based assessment takes many forms and can include the four methods 
included in the brief review above. Scenario-based assessments, surveys, observa- 
tion methods, and portfolio designs can all be used to structure assessment tasks and 
their scoring criteria within technology environments. 

Computer-based applications may have additional advantages over other media 
for assessment because they can provide automated scoring and reporting. In a 
study conducted by Harahap et al. (2020), a digitalized meta-inquiry assessment 
helped students to receive rapid feedback at each of the stages including problem 
formulation, hypothesis development, data collection, hypothesis testing and over- 
all research progress. The social system component in the model allowed students 
to maintain continuous interactions with teachers and cooperation with colleagues 
as peer mentors, mediators, and as motivators. 
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Technology-based assessment has been found to be cost-effective and environ- 
mentally friendly because the entire process of inputting responses, scoring, and 
giving feedback is paperless. Students can respond to questions more quickly than 
when working with paper and pencil assessment versions (Khairil & Mokshein, 
2018). Beyond such facilities as tracking of speed which for assessment in some 
contexts is valued, assessment of skills such as problem-solving has highlighted the 
fact that multiple paths through tasks can be tracked, hence enabling other charac- 
teristics of these skills to be prioritised (Ramalingam et al., 2017). This tracking 
facility is a characteristic that lends itself well to the assessment of twenty-first 
century skills. Since many of these skills are regarded as complex in nature and in 
demonstration, the facility of tracking branching within a skills demonstration envi- 
ronment is a major benefit. 

Some researchers have identified limitations associated with the use of computer- 
based assessment. Schaeffer and Palmgren (2017) postulated that activities that are 
ambiguous may fail to transform into assessment frameworks that require stan- 
dardisation in order for data capture to take place seamlessly. There are also con- 
cerns related to inequality when using technology-based assessment. Due to 
economic variations within and among different locations and schools, some indi- 
viduals may have more access to devices and greater computer literacy skills than 
others, thus leading to disparities in performance. Similarly, assessment conditions 
and procedures, such as internet connection speeds, power fluctuations and hard- 
ware and software specifications, tend to vary from one location to another and 
these may directly affect the performance of some students (Blazer, 2010). 


7.4 Overview of Assessments of Twenty-First Century Skills 
Used in Sub-Saharan Africa 


In order to identify existing assessments of twenty-first century skills that are being 
used in SSA, the authors conducted a review of assessments. Assessments were 
identified by searching electronic databases, scanning reference lists, and consulta- 
tions with experts in the field of measurement, assessment, and evaluation. Google 
Scholar and platforms such as INEE? Measurement Library and Research Gate were 
used to gather a pool of assessments of twenty-first century skills. The criterion 
applied was that the assessments were used in the region, rather than whether they 
were developed specifically within or for the context. Assessments were considered 
only if they focused on twenty-first century skills, had documented technical infor- 
mation, and were developed for children or adolescents. In addition, tools that had 
a commercial cost associated with the purchase and administration of the assess- 
ment were not considered. 


?Inter-agency Network for Education in Emergencies (INEE) 
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Based on these criteria, seven assessment were identified and examined along 
five dimensions. The dimensions are assessment purpose, assessment type or form, 
assessment context, target population, and skills assessed. These are described 
below. There are many dimensions upon which reviews of assessments can be based 
(see Galloway et al., 2017; Lamb et al., 2019; Soland et al., 2013). Given the imple- 
mentation environment for ALIVE, the five selected dimensions provide a grounded 
approach to reviewing tools in the context of SSA. The interdependence of decision- 
making about each of these dimensions is highlighted in the brief descriptions. 


7.4.1 Five Dimensions 
7.4.1.1 Assessment Purpose 


According to Schwartz et al. (2011), there are five broad purposes of assessments, 
including monitoring system performance, accountability, setting priorities and sig- 
naling important competencies, and supporting instructional improvement. The 
more general identification of assessment for summative or formative purposes 
overlays this. The purpose is formative if the assessment will be used to support 
teachers in setting and reviewing instructional goals and providing individualized 
feedback to students. The purpose is summative if the assessment is used to deter- 
mine whether an instruction has been effective after the fact, and data can be pro- 
vided to the individual, school, or system. Some assessment tools are constructed in 
ways that lend their use to both purposes. The need for detail in assessment results 
is determined by purpose, with individual formative typically needing more detail 
than large scale benchmarking or accountability. 


7.4.1.2 Assessment Type or Form 


Five approaches to measuring twenty-first century skills have been described. These 
are scenario-based assessment, questionnaires/self-report, video recording and 
direct observations, portfolio assessment, and technology-based assessment. Some 
measuring tools draw on more than one approach in order to capture multiple facets 
of the target constructs. This strategy can elicit rich data but can also present some 
analytic and reporting challenges. For example, mixing both survey self-report for- 
mats together with scenario-based formats presents challenges for data analysis 
(Hoskins & Liu, 2019). The context or physical environment in which the assess- 
ment is to take place must be considered in decisions about the type or form of 
assessment to be used. Where a household-based assessment is involved, fewer 
resources will typically be available to draw on for an assessment event than in a 
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classroom. The availability of resources influences how complex the type or form of 
assessment can be. 


7.4.1.3 Context for Use 


The context in which the assessment takes place will determine the type or form of 
assessment. The main context for use of educational assessments is the classroom. 
This provides the facility for assessing large numbers of individuals in a relatively 
efficient and standardised way. However, it also prioritises the use of the written 
word or technologies in order to enable group administration. Other contexts include 
out-of-school programs, which may offer a mix of administration facilities; and 
household-based assessment, which is typically less well-resourced. Household- 
based assessment has been used in citizen-led approaches to gather large-scale data 
for advocacy purposes. Each of these contexts offers variably beneficial environ- 
ments for the assessment of skills like literacy and numeracy, or for assessment of 
skills such as communication or creative thinking. 


7.4.1.4 Target Population 


The nature of the target population establishes the possible type of assessment, its 
context, and the actual target constructs. The maturational level of the individuals 
and groups will determine what characteristics or capabilities can be assessed, and 
through what medium. Significant aspects will include whether language and liter- 
acy skills can be assumed; and elements such as concentration span, physical com- 
fort, and communication modes will be considered. The capacity of the target 
population to consent to assessment is an important feature, particularly where the 
assessment will take place outside of the formal education context. 


7.4.1.5 Specific Skills 


The selection of skills to assess has a greater influence on decisions on all other 
dimensions. Different skills require different modalities for expression, and so will 
determine the type or form of assessment. Since many twenty-first century skills, as 
discussed, are complex, it may be necessary to accept limitations in their capture. 
Accordingly, some aspects of a complex skill may be targeted while others will be 
put aside due to the overwhelming difficulties in collecting evidence. Such deci- 
sions are pragmatic, and as long as they are made explicit, can be justified in report- 
ing results. 
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7.4.2 Selected Assessments 


7.4.2.1 Assessment of Life Skills and Values in East Africa (ALIVE; 
Mugo, 2024) 


ALIVE is designed to measure life skills and values of adolescents across Kenya, 
Tanzania, and Uganda. Tools were developed to assess collaboration, problem- 
solving, respect, and self-awareness of 13-17 year-old adolescents through the 
medium of household-based assessment. The goal of ALIVE is to generate evidence 
of these proficiencies for use by governments in their current efforts to include these 
competencies in the education system. The tools use scenario and performance- 
based tasks, scored by qualitative rubrics. The assessments are administered orally 
on an individual basis apart from the collaboration tasks which are administered to 
groups of four adolescents. 


7.4.2.2 Optimizing Assessment for All (OAA; Kim & Care, 2020) 


OAA was a collaborative effort designed to strengthen education systems’ capacity 
to integrate twenty-first century skills into their teaching and learning through use 
of assessment. Assessments for collaboration, problem solving, and critical think- 
ing were integrated within mathematics, health, environment, English, social stud- 
ies, and science for grades 4-8. OAA is a set of classroom assessment tasks with 
both closed and open response types that are scored using rubrics. OAA assessment 
tasks took traditional test items and revised them to assess complex skills aligned 
with each country’s new learning goals. A critical product of OAA was generic 
templates which allow for the development of assessment tasks that take into 
account culture, context, skills of interest, and can be created in any country using 
their existing items. 


7.4.2.3  ISELA (D’Sa & Krupar, 2019) 


The International Social and Emotional Learning Assessment (ISELA) was devel- 
oped among Syrian refugee children in Kurdistan, Iraq, to understand self-concept, 
stress management, perseverance, empathy, and conflict resolution in children of 
6-12 years. The context was conflict-affected and fragile states, and so the tool 
design targeted understanding childhood resilience. The purpose of tool develop- 
ment was for use in program and national evaluation. The focus is on social- 
emotional skills acting as protective factors. 
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7.4.2.4 Educator Assessment of Learners’ Soft Skills Ability (EALSA; 
Education Development Center, 2019) 


The EALSA is primarily a work readiness tool developed within the context of the 
Measuring Skills @ Scale Project. The tool is focused on the skills of communica- 
tion, interpersonal skills, dependability, and problem solving/critical thinking. It 
comprises a self-report questionnaire using items that in the main are contextual- 
ised within the work environment. The purpose is to generate evidence of respon- 
dents’ current levels of functioning and so provide information to instructors about 
where best to intervene to improve performance. 


7.4.2.5 The Youth Power Action Youth Soft Skills Assessment (YAYSSA; 
Omoeva et al., 2023) 


YAYSSA measures social and emotional skills of youth in low-resource settings. 
Specific targets are positive and negative self-concept, higher order thinking skills, 
and social and communication skills. The tool is designed for use in evaluation of 
programs to improve economic outcomes and functioning of youth. It is a self- 
report tool, with a mix of attitudinal responses to vignettes, and self-evaluation of 
the individuals’ skills. 


7.4.2.6 Life Skills and Citizenship Education Measure (LSCE; Hoskins & 
Liu, 2019) 


The LSCE is a product of a collaboration of UNICEF and the World Bank in the 
Middle East, with the tool developed to measure 12 life skills in the context of Life 
Skills and Citizenship Education. The goal is to use the information to inform gov- 
ernment priorities, policy, and instructional objectives. The current tool captures 
eight of the skills through a mix of scenario-based multiple-choice response formats 
and self-report of competencies using Likert scales. 


7.4.2.7 Tanzania Life Skills Assessment (Research Triangle 
Institute, 2019) 


The Tanzania Life Skills Assessment takes a self-report approach to its item design 
for grit and self-control, and a performance-based approach to problem solving 
which collects a number of methods respondents use to solve word problems associ- 
ated with EGRA. Based on listening to brief scenarios, the respondent identifies 
how likely they are to behave in the same way as the protagonist. 
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7.4.2.8 Review of Selected Assessments 
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Each of the seven assessments was reviewed in light of the foregoing description of 
common methods used in the assessment of twenty-first century skills, and the five 
dimensions. Table 7.1 provides a summary of information for each assessment. 
Note that the inclusion of these assessments is neither an endorsement nor confir- 
mation that they are reliable and valid. 


Table 7.1 Selected assessments of twenty-first century skills in sub-Saharan Africa 


Assessment Purpose Type/Form Context Population | Target skills 
Assessment of | Summative: | Scenario- Household | Adolescents | Problem solving, 
Life Skill and | setting based and aged self-awareness, 
Values in East | policy performance- 13-17 years | collaboration, and 
Africa priorities based with respect 

(ALIVE) scoring rubrics 

Optimizing Formative | Scenario- School Children in | Problem solving and 

Assessment for based with grades 4-8 | collaboration 

All (OAA) scoring rubrics 

International Summative: | Vignettes; School Children Social-emotional: 

Social and program performance- aged self-concept, stress 

Emotional monitoring | based 6-12 years | management, 

Learning perseverance, 

Assessment empathy, 

(ISELA) relationship 
management, and 
conflict resolution 

Educator Formative | Multiple School; Students in | Work readiness: 

Assessment of choice, program | secondary communication, 

Learners’ Soft scenario-based and tertiary | interpersonal skills, 

Skills Ability questions dependability, and 

(EALSA) problem solving/ 
critical thinking 

Youth Power | Summative: | Self-report School; Youth aged | Self-concept, 

Action Youth | Program program 15-19 years | higher-order 

Soft Skills evaluation thinking skills, 

Assessment social and 

(YAYSSA) communication 
skills 

The Life Skills | Summative: | Multiple School Grade 7 Creativity, 

and Citizenship | setting choice, children problem solving, 

Education policy scenario-based negotiation, 

(LSCE) priorities questions; decision-making, 

measurement self-report self-management, 

instrument respect for diversity, 
empathy, and 
participation 

Tanzania Life | Summative | Self-report; School Late primary | Academic grit, 

Skills some school-aged | self-control, and 

Assessment performance- children problem solving 


based 
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Most of the seven tools capture information for summative purposes, with just 
two of these designed to inform policy. The two tools that are designed to inform the 
education sector most consequentially are derived from two quite different sectors, 
but both draw on strong government support. ALiVE and LSCE are both large-scale 
initiatives, one supported primarily by the civil organisation sector, and the other 
supported by UNICEF and the World Bank. Their target skills draw on the cogni- 
tive, interpersonal, and intrapersonal, including competencies associated with 
global competence and citizenship. With the exception of EALSA and its focus on 
work readiness, and OAA and its classroom focus, the remaining tools prioritise 
skills that might act as protective factors or skills of interest in the face of conflict 
and adversity. In terms of method, self-report remains the dominant method of col- 
lection of information. Given the dominant use of this method, the contexts in which 
several of the tools were developed, and the age of samples that contributed data, 
there may be concern about the degree to which respondents were at a level of cog- 
nitive or social-emotional maturity to able to provide data truly symptomatic of 
authentic actions in a real world. 


7.5 Conclusion 


Sub-Saharan African countries are increasingly adopting twenty-first century skills 
or life skills into their primary and secondary education. However, in the main, 
assessment remains focused on traditional academic-oriented curricula. As national 
education systems move more rapidly into nurturing and assessing new competen- 
cies, there are few resources available to draw from. 

Methods currently used in the region have not been reviewed critically to stimu- 
late change in the approach to assessment of these dynamic competencies. This 
chapter focused on identifying some assessment approaches for twenty-first century 
skills applied in SSA. The form of assessment tools found most commonly used in 
SSA for measurement of these skills are scenario-based assessment with standard- 
ized scoring rubrics, and self-report questionnaires using Likert scale response 
options. While self-report questionnaires are favoured for their simplicity in design, 
administration and scoring, there are serious concerns about the degree to which 
they stimulate socio-culturally desired responses. Scenario-based activities how- 
ever, have gained recognition for their ability to prompt respondents into multidi- 
mensional action or thinking, fostering idea generation without fear of presenting 
incorrect responses. An example of this approach, from East Africa, is the ALIVE 
project, which employed a context/scenario-based assessment method to gauge 
adolescents’ proficiency in specific competencies. The outcomes of this assessment 
furnished compelling evidence supporting the method’s effectiveness. Furthermore, 
the robustness of the scoring approach adopted facilitated useful behavioural dif- 
ferentiation across proficiency levels, providing information that could be used for 
instructional purposes by teachers. 
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The review of tools in this chapter highlights the relative paucity of diversity in 
tools development in the SSA region. This finding alone is useful in guiding future 
efforts to experiment with multiple approaches rather than defaulting to approaches 
that have been used traditionally to capture evidence of achievement in subject- 
matter in the education sector. 
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Abstract Household-based assessments (HBA) in education are novel. Conducting 
household-based learning assessments has long been associated with non- 
government organisations as evidenced in India, other parts of South Asia, East, 
West and Southern Africa, and the Americas. The use of HBA to measure life skills 
expands the use of this approach that was long characterised by assessing founda- 
tional literacy and numeracy skills. Experiences from the Assessment of Life Skills 
and Values (AliVE) by the Regional Educational Learning Initiative (RELI) reveal 
that opportunities for using HBA to measure life skills are immense. Through a col- 
laborative initiative, ALIVE as a process delivered a tool that was used across 
Kenya, Tanzania, and Uganda mobilising stakeholders among policymakers, educa- 
tion researchers, teachers, civil society organisation leaders, local partner organisa- 
tions and citizen assessors from areas where a sample of adolescents aged 13-17 
was identified, selected and assessed on the value of respect and life skills of self- 
awareness, problem solving and collaboration. This household-based assessment 
approach reveals that despite the immense opportunities, there are also challenges. 
This chapter provides the context of ALIVE, traces the process of implementing 
ALIVE using the household-based approach, and discusses the opportunities and 
challenges associated with using HBA in measuring life skills. These opportunities 
include the ability to capture a larger range of children than can school-based assess- 
ments, the liberalisation of assessments, the inbuilt advocacy and technology oppor- 
tunities, the deepening of external accountability systems, and engagement 
capabilities. Among the challenges discussed in using HBA for measuring life skills 
is the need to identify and define the purpose, the complexity of identifying groups 
to assess, the logistical challenges in implementation and associated costs, difficul- 
ties in scheduling, and the reality that still not all children will be reached. Despite 
this set of challenges, HBA demonstrate the opportunities that exist outside school 
to provide data at large scale for reporting and advocacy. 
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8.1 Introduction 


Nansubuga et al. (2024) provide a detailed description of methods available for 
assessing life skills. In this chapter, the focus is on the opportunities and challenges 
of using household-based assessments (HBA) for measuring life skills, drawing on 
the experience of applying the approach in the context of the Assessment of Life 
Skills and Values in East Africa (ALIVE) project. The chapter traces the origins of 
household-based learning assessments over the past one and a half decades when 
the approach emerged, principles and standards developed, methods defined, assess- 
ments created, and lessons learned, before delving into use of the approach in 
assessing life skills and values. 

In this chapter, five critical processes in the ALIVE process that define its rigour 
(sampling, tools development, selection of assessors, training, and the actual assess- 
ment activity) are presented in detail to situate the opportunities that HBA bring to 
measuring life skills. HBA in education are recent strategies used for estimating the 
education system’s efficacy, effectiveness, and efficiency in resource application. 
Although household-based surveys are a common epistemological approach in 
social sciences, the household as a point of identifying and assessing learning is 
relatively recent. The most documented HBA, also considered the prototype of 
household-based learning assessment in education, is Pratham’s Annual Status of 
Education Report (ASER, 2005). 

Pratham, one of the largest education non-government organisations in India that 
had been working across the informal settlements of Mumbai City, developed an 
innovative approach to establishing levels of foundational learning skills among 
children of school-going age. Motivated by the realisation that not all children of 
school-going age were actually in school, Pratham decided to use the household as 
the point of collecting the data. The Pratham assessment therefore harvested data on 
learning levels for out-of-school children as well as school attendees. Taking an age 
group of 6-16 years as the assessment group, the criterion-referenced learning 
assessment pegged on Grade 2 reading and numeracy expectations was the first step 
in universalising assessments at large scale. Citizen volunteers in their thousands 
were recruited from the sampled villages, trained, and deployed back in their com- 
munities (hamlets and villages), to sample-based identified 20 households within 
each, recorded details of all children in the target age group, and surveyed the chil- 
dren aged 6-16 regardless of their schooling status. The deployment of citizen vol- 
unteers for the complex task of assessing learning at household level, also led to 
referring to these assessments as citizen-led assessments. Sometimes the two termi- 
nologies i.e. household-based and citizen-led assessment have been used inter- 
changeably or simultaneously (Nakabugo, 2016, 2021). The initial ASER citizen-led 
HBA in 2005 sampled more than 600 districts across India (Goodnight & 
Bobde, 2018). 
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The ASER approach would later spread across South Asia starting with Pakistan, 
later in the East African countries of Kenya, Tanzania, and Uganda, West Africa, 
and Latin America, in what became a horizontal transfer of learning across the 
global south. The East African initiative, named Uwezo (meaning ‘capability’), 
conducted its first pilot assessment in Kenya in 2009. Following the success of this 
pilot, the assessment was rolled out in Uganda and Tanzania in 2010, and other 
countries in Africa such as Mali and Nigeria, among others, also embraced the 
approach. By March 2020, citizen-led HBA had been embraced fully in 15 coun- 
tries on three continents (Table 8.1) and had assessed over 8.5 million children in 
literacy and numeracy in over 30 languages including English, French, and 
Portuguese. 

The ASER transfer maintained the original approach, but with adaptation of 
tools to meet the demands of each context (Mugo et al., 2016). The ASER tools and 
approach would later be called a method of education system feedback and program 
evaluation. The approach gained new names as adjustments were made to its use (as 
indicated in Table 8.1) while processes were adopted among them retaining the age 
of assessment, use of citizen assessors (and criteria such as levels of education), 
tools development, assessing at the household, retention of reading and numeracy as 
basic competencies to assess, and administration of instant feedback as essential 
elements in the assessments (Nakabugo, 2021). 


Table 8.1 Citizen-led assessment/household-based assessment initiatives globally (as of 
March 2020) 


Year of first [pilot] 
Name of the initiative Country assessment 
ASER Centre India 2005 
ASER Pakistan Pakistan 2008 
Uwezo Kenya (now operating as Usawa Agenda) Kenya 
Uwezo Tanzania Tanzania 2010 
Uwezo Uganda Uganda 2010 
Beekunko Mali 2011 
Jangandoo Senegal 2012 
MIA (Medición Independiente de Aprendizajes - Mexico 2014 
‘independent assessment of learning’) 
LEARNigeria Nigeria 2015 
IID (Institute of Informatics and Development) and Bangladesh 2015 
BRAC 
TPC Mozambique Mozambique 2016 
ASER Nepal Nepal 2016 
VIdA (Valoración Intersubjetiva del Aprender — Nicaragua 2017 
‘intersubjective assessment of learning’) 


Source: People’s Action for Learning (PAL) Network: http://palnetwork.org/ In Nakabugo (2021, 
p. 51) 
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In the adaptation of the approach, generic steps required in large-scale assess- 
ment were taken, including: defining the purposes of the assessments that make 
them fit for purpose; definition of skills and competencies to be assessed (including 
defining rubrics as performance indicators); and establishing the elasticity of the 
tools across diverse targeted groups (through the iteration of several field trials), 
before the tools were deployed for use. This collaborative approach drew on cur- 
riculum development and assessment agencies in respective countries, and on 
researchers, psychometricians, and practising teachers to ensure rigour and quality 
(Mgalla et al., 2020). Since the assessments are household-based and within com- 
munities, the selection of assessors is a critical issue. According to PAL Network 
(2021), working with citizen volunteers requires the development of criteria that 
include identifying volunteers from communities where they are likely to work, and 
establishing their thresholds of education. For the former, the assumption is that by 
selecting volunteers (surveyors) who hail from their community, these are likely to 
participate in collectivising citizen agency, that is, citizens taking collective action 
to find solutions to the learning challenge, a key ingredient in the design and imple- 
mentation of the citizen-led bottom-up approach to improving learning outcomes 
(Goodnight & Bobde, 2018). 

Wiles et al. (2013) argue that when applying innovative approaches in research, 
uptake and acceptance of these new approaches and their findings in the broader 
social science community may often be slow. Yet, although HBA are novel, their 
implementation and role in altering the education landscape cannot be underesti- 
mated. The ‘World Development Report: Learning to realise the promise of educa- 
tion’ (World Bank, 2018) frames the learning poverty phenomenon based on data 
from these HBA. 

It is because of the potential advantages of the household-based approach to 
learning assessment and its global acceptability as a credible research approach, that 
the Uwezo HBA approach that had been adapted in the East African context was 
embraced in ALiVE. 


8.2 The ALIVE Study 


In 2019, the Regional Education Learning Initiative (RELI) (an initiative of civil 
society organisations in East Africa) coalescing around its values and life skills 
thematic group, began consolidating experiences, literature, and practice of life 
skills and values in education across the East African countries of Kenya, Tanzania, 
and Uganda. This led to the development of a contextualised tool for measuring the 
selected life skills of collaboration, problem solving, and self-awareness, and the 
value of respect. HBA was undertaken across the three countries between April and 
August 2022, with its results reported in Ariapa et al. (2024). ALIVE’s rigour is 
drawn from five distinct processes of sampling, tools development, recruitment of 
assessors (citizen volunteers and teacher trainees), training and data collection 
methods/actual assessment. 
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8.2.1 Sampling 


The HBA approach involved drawing a population sample to assess life skills and 
values among adolescents aged 13-17 years who were both in and out of school. 
The assessment was conducted with a geographically representative sample of sub- 
national units (districts/counties) in each country, with 20 enumeration areas in each 
sub-national unit, and 20 households with adolescents selected to participate in the 
assessment. The ALIVE sampling was a three-stage process. 


8.2.1.1 Sampling Stage 1 


This stage involved the selection of the sub-national units and enumeration areas to 
be visited. The relevant national Bureaus of Statistics undertook this task. The 
Bureaus identified the sub-national units from each national census frame and then 
proceeded to identify 20 enumeration areas (EA) in the respective sub-national units. 


8.2.1.2 Sampling Stage 2 


This stage involved the physical identification of the selected enumeration areas and 
selection of households to be visited. ALIVE, being a collaborative undertaking, 
involved the use of sub-national partners based in their respective areas. The part- 
ners were mostly drawn from organisations which had previously conducted HBA 
(e.g. Uwezo learning assessments) in their respective areas. The selection of the 
households was supported by technology. Upon the liaison team (a district coordi- 
nator and two village coordinators) reaching the EA, they worked with relevant 
local government administration officers to identify the boundaries of the 
EA. Subsequently, they walked through the entire EA to list households with ado- 
lescents. This task, identified as household listing, used an online form that captured 
basic information about the name of the household head, number of the adolescents 
regularly living in the household, and a description of the household heads. These 
lists were collected and 20 households for each EA were sampled randomly. 


8.2.1.3 Sampling Stage 3 


This stage involved the selection of the adolescents to be assessed. Households listed 
were those found to have adolescents. Once the assessors reached the households and 
introduced themselves, they re-established the number of adolescents in the house- 
holds and selected two adolescents from five scenarios: (i) households with one ado- 
lescent (ii) households with two adolescents of either gender (iii) households with two 
adolescents of different gender (iv) households with more than two adolescents of the 
same gender (v) households with more than two adolescents of different gender. 
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8.2.2 Tools Development 


ALIVE embraced a collaborative process of contextualising tools development 
for the assessment. This involved setting up a technical team drawn from the 
project core team members (from the three countries), experts from universities, 
practicing teachers, and officers from curriculum and assessment bodies from 
the respective countries. The team under the guidance of an external adviser 
identified the skills and values to assess, developed a skills framework, drafted 
items (tasks), pre-tested them, and piloted them before constituting a panel that 
assembled the final assessment tool. As the assessment was to be conducted at 
household level, was to involve in-school and out-of-school adolescents, and 
was to be conducted by non-conventional assessors (citizen volunteers and 
teacher trainees), extra attention was given to ensuring that the tasks were sim- 
ple and could easily be understood by the assessors and adolescents and that 
they were relevant and acceptable in the local context. Pre-testing and piloting 
the tool in different social and cultural contexts helped in phrasing the assess- 
ment tasks appropriately. 


8.2.3 Recruitment of Assessors/Enumerators 


Since ALIVE was to be conducted at household level (not at school) and within 
communities, the selection of assessors was a critical issue. Given the more com- 
plex nature of the ALIVE tools compared to those assessing basic literacy and 
numeracy, the minimum education qualification of the assessors was set to a mini- 
mum of 6 years of secondary education (Advanced Level) for the citizen volunteers. 
Similar to Uwezo, ALiVE collaborated with district-based local organisations that 
mobilised qualified volunteers to carry out the assessment and obtain other relevant 
data across the householdsand community settings. Calls for volunteer posters were 
displayed in sampled villages to recruit competent persons living within the com- 
munities to undertake the assessment activities. 

In Kenya and Uganda some assessors were teacher trainees who were pursuing a 
teaching qualification at diploma or degree level. These were selected through part- 
nering or direct contacts with the selected teacher training institutions that were 
located in the sampled assessment districts. It was also important to select assessors 
who were ethically in good standing in the community or faculty. Hence, validation 
of this attribute, such as a recommendation from the local or teacher training insti- 
tutional leadership, was critical for purposes of child safeguarding and credibility of 
the assessment process. The language of the assessor was also a consideration in the 
recruitment of the enumerators. While assessing at the household level, it was 
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important that the assessor was able to administer the tasks in a local language 
widely spoken in the area in case the adolescent being assessed opted for that. This 
was to ensure that the adolescents’ responses to the tasks was not constrained by 
language barriers. 


8.2.4 Training of Enumerators 


A cascaded mode of training as is usually done with Uwezo learning assessments 
was applied to ALiVE. The first step was trainer recruitment from the existing pool 
of Uwezo trainers to supplement staff from ALiVE implementing organisations 
who had participated in the technical process of developing the assessment tools. 
Together with district coordinators, these constituted a team of trainers who would 
then train the enumerators at district level. The trainers underwent a 3-day training 
of trainers that was conducted by a select team of Master Trainers and staff from 
ALiVE implementing organisations. Those newly trained were then deployed in 
pairs to allocated districts, supported by two to three Village Coordinators, to con- 
duct volunteer trainings. The volunteers were prepared through familiarisation with 
ALIVE, the assessment process, entry into the community, and acquired the neces- 
sary skills to assess, grade and record the assessment results for all adolescents, and 
abide by set safeguards and ethical standards. As noted above, selected trainers and 
district coordinators who followed a well-structured training manual designed by 
ALIVE, facilitated the district-based volunteer training. 


8.2.5 Data Collection 


Decisions on when to conduct data collection were devolved to respective countries. 
The timing of the assessments was aligned with the expected periods during which 
adolescents would be at home, considering that most adolescents were of school- 
going age and therefore would only be home when schools are closed. 

In each household, up to two adolescents, preferably a male and a female, were 
selected and assessed in the selected life skills of problem solving and self-awareness 
and the value of respect. Each of these adolescents also undertook a literacy assess- 
ment pegged on the Grade 4 curriculum in their respective countries. Additionally, 
three groups of four adolescents per group in each EA (a male only, female only and 
mixed group) were identified and assessed on the life skill of collaboration. This 
process provided rich experiences that offer learning on the challenges and oppor- 
tunities of using HBA to measure life skills. 
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8.3 Factors in Use of Household Based Assessment 
for Measuring Life Skills and Values 


8.3.1 Opportunities and Challenges Presented by 
Household-Based Assessment 


The opportunities and challenges associated with using HBA are many and varied. 
Some of these are discussed in this section, informed by research studies and by the 
ALIVE experience. 

Muñiz and Fonseca-Pedro (2019) list ten steps in assessment, Downing (2006) 
lists 12, and Foertsch (2014) lists five steps for effective test development 
(Table 8.2). 

Analysis of these steps reveals the phases of pre-development, development, and 
post-development. The ALiVE process included the steps listed in Table 8.2 as well 
as some additional development activities. ALiVE included an extensive literature 
review; a contextualisation study of how the community regarded life skills and 
values leading to the contextual definition of the selected constructs (life skills and 
value); hypothesising the constructs’ structures; drafting of tasks and accompanying 
rubrics; paneling of materials and revising the tools based on the responses 


Table 8.2 Steps in assessment 


Muñiz and Fonseca-Pedro | 


(2019) Downing (2006) | Foertsch (2014) 
1. Stating the problem 1. Overall plan 1. Define traits 
2. Writing specifications | 2. Content definition 2. Define test characteristics 
for the test 3. Test specifications: 3. Develop/identify set of items 
3. Writing and Blueprinting the test 4. Obtain item statistics 


moderating items . Item development 5. Finalise specifications and 


4 
4. Informal trialling of 5. Test design and assembly procedures 
items on native 6. Test production 
speakers 7. Test administration 
5. Trialling of the test on | 8. Scoring examination 
a group of non-native responses 
speakers similar to 9. Establishing passing scores 
those for whom the 10. Reporting examination 
test is intended results 


6. Analysis of results of | 11. Item banking 
the trial; making of 12. Test technical report 
any necessary changes 
7. Calibration of scales 
. Validation 
9. Writing handbooks 
for test takers, test 
users and staff 
10. Training staff 


oo 
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collected; field trials of the tools; finalisation of tool; assessment activity; scoring 
and reporting. 


8.3.2 Opportunities 


Five reflections on the opportunities that HBA bring in assessing life skills are 
shared, drawing on the ALIVE experience. 


Opportunity One: HBA can capture a larger range of children than can school-based 
assessments 


Using HBA as distinct from school-based assessment enabled ALIVE to access a 
wide range of adolescents, which suited ALiVE’s target population that was age- 
based rather than grade-based. ALiVE’s target population was adolescents aged 
13-17 years irrespective of their schooling status. These included the out-of-school 
adolescents occasioned by three circumstances. First, there are adolescents who 
never enrolled in school. Second, there are adolescents who have completed a par- 
ticular level of education and have not proceeded to the next. Third, there are ado- 
lescents who have dropped out of school. When assessments are school-based, these 
categories of adolescents would be disenfranchised since they are available only in 
the community spaces. It is this capacity that makes HBA a valuable facilitator in 
moving towards universalising data gathering that informs national informa- 
tion needs. 


Opportunity Two: HBA liberalise assessments by enabling non-government agencies to 
convene and coordinate assessment efforts 


Assessments have long been associated with formal education settings. Expertise, 
whether in or outside governments, mostly focuses on the school where mobilisa- 
tion of learners is easy. Equally, approval processes for school-based assessments 
are faster to enable than assessments conducted outside the school and where mul- 
tiple players are involved. Hence, there is little evidence of policy-driven assess- 
ments that have been conducted outside the school setting. 

ALIVE demonstrates the convening power of non-government agencies. Through 
collaboration, ALIVE mobilised actors which included policy agencies (curriculum 
development and assessment bodies), practicing teachers, academia, and project 
managers to collaboratively design and implement an assessment around areas con- 
sidered to be complex to assess. To a large extent, it can be argued that the conven- 
ing power of the assessment rested with the non-government sector. Perhaps this 
was possible only because assessments were outside of the formal education system 
where no policy agency could claim to mark its territory. Although framed as an 
assessment, ALIVE cuts across sectors allowing for a sector-wide approach to plan- 
ning and designing, including multiple players at both national and sub-national 
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levels, use of citizen assessors (not assessment experts let alone practicing teach- 
ers), and local administrators; all in the context of bringing assessment right into the 
household. This reveals the power that non-government organisations can have in 
convening to encourage complex policy processes. 


Opportunity Three: Inbuilt advocacy is an ingredient of HBA 


HBA have been designed specifically to engage with key education stakeholders at 
all levels, right from the smallest unit of society, the household unit, up to the 
national policy-making level in Ministries of Education. Nakabugo (2016) shares 
experiences on the Uwezo studies’ influence on policy that has been punctuated 
with deliberate and early buy-in from stakeholders. Right from inception and simi- 
lar to Uwezo assessment purposes, ALiVE was not only focused on collecting data 
on life skills and values but also on using the assessment evidence to trigger a 
national conversation at all levels to raise awareness of the need for all young people 
to acquire these skills and values. Hence, in agreement with Mayne et al. (2018) 
who argue that the data generation process for policy-based advocacy processes is 
as important as the data itself, the process was equally important for ALiVE. As 
such, policymakers and other key actors in the system, including parents and care- 
takers, ought to be part of the evidence generation and not be recipients at the tail 
end. The household-based approach enabled this to happen. Policy actors at national 
level were involved in the ALiVE assessment process either as members of the tech- 
nical team that developed the assessment tools or as members of the Advisory 
Committee that was regularly consulted on the assessment plans or as observers of 
the actual assessment process. 

Often, data generators exclude input from end users. As such, advocacy is left 
with responding to two issues; defending the method followed, and justifying the 
content of the findings. When findings are shocking, users look out for ways to 
question the evidence. And one such fallback position is often to query the method 
used in generating the data. Through co-creation, evidence users can be mapped 
early in the process, and spaces can be deliberately created for the users to interact 
with the process as the evidence is built. This was the case with the ALIVE assess- 
ment process. Ultimately, the users develop a deep understanding of the process and 
are then freed to deal with the evidence without doubting its source. This co-creation 
approach has been argued to deliver better advocacy results (Avermaet & 
Shohamy, 2022). 

In the case of ALIVE, the co-creation approach adopted through the convening 
power of non-government efforts allowed for multiple partners to plug in to define 
the life skills, develop the skills structure, draft tasks, subject them to pre-tests, 
pilot, and assemble them before rolling out the assessment. At each point, the inclu- 
sion of different partners, organisations, policy institutions, and individuals helped 
to clarify questions about the definitions, skills structure, items, and assessment 
processes. This allowed learning to take place in real time. Ultimately, one may 
argue that ALIVE served the advocacy purposes of creating awareness around life 
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skills even while still engaged in generating the target illustrative evidence on the 
status of life skills among adolescents in East Africa. That the assignment was con- 
ducted at the household level meant that all the parents of the over 45,000 adoles- 
cents were sensitised on life skills. At each household, a message on nurturing life 
skills was left through a calendar containing a checklist of six things that a parent/ 
caretaker could undertake to nurture life skills for the adolescents. This reveals the 
inbuilt advocacy power of HBA generating early positive effects. Since key actors 
were involved in the assessment process at different stages, one key stakeholder 
linked to a national assessment agency noted at the launch of the ALIVE report in 
Uganda in December 2022: “This assessment and its findings are timely. As we are 
in the process of preparing assessment tools for the first candidates going through 
the new lower secondary curriculum, we will invite ALiVE team to contribute to 
this process”. This level of acceptability was mainly possible because the official 
had interacted with the assessment process as a member of ALiVE Advisory Group. 


Opportunity Four: Technology is a facilitator of measuring life skills in HBA 


Use of Information and Communications Technology (ICT) in research has become 
more of a necessity than an option in contemporary research due to its advantages 
in easing administration, record keeping, capture, storage and analysis of data, and 
in report writing. Similarly, the ALiVE collaborative process benefited immensely 
from the availability of different technologies before, during, and after the assess- 
ment. Because of the many processes involved in executing HBA, the critical role of 
ICT in research was magnified. Before the assessment, the second stage of sampling 
(listing of households) was loaded on an Android enabled app — KoboCollect. This 
allowed for the collection of basic information on the eligible households for listing 
(households with at least one adolescent and the bio-details of the household head). 
During the assessment, the same technology was used by the assessors to read out 
the task and item prompts to the adolescent, and to record response codes. The 
inbuilt features of the tools allowed for skip logic (increasing data reliability by 
reducing errors). With data being recorded using technology, it was possible to pro- 
cess the data script, enabling data recheck before data collection was completed. 
The choice of such applications which allow for storage of data before being 
uploaded allows for iteration where corrections need to be made, on any smart- 
phone, making them easier and cost-effective to use. As such, the turnaround for 
data submission is shortened and processing reports can take a shorter time. This 
demonstrates that HBA facilitated by technology is a feasible pathway to pursue 
when assessing life skills even in East African contexts. 


Opportunity Five: Using HBA to measure life skills may deepen external accountability for 
school systems 


School systems are the universal pathway to the acquisition of knowledge, skills, 
and attitudes necessary for societies to thrive. Ideally, schooling is available through- 
out childhood. Some school systems begin admitting and enrolling children as early 
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as 3 years and children exit secondary schools on the eve of adulthood. Within East 
Africa, the primary school starting age is 6 years in Kenya, Tanzania, and Uganda. 
With 12-13 year education systems, learners exiting the school are young adults. In 
some cases, the time in education varies due to delayed enrolment and grade repeti- 
tion (a common phenomenon associated with resolving the challenge of delayed 
acquisition of grade-specific skills and competencies within cohorts). 

Recent evidence indicates the prevalence of learning poverty, with more children 
reaching age 10 years without mastering foundational learning skills such as liter- 
acy and numeracy (World Bank, 2021, July). In an assessment of functional systems 
that realise better outcomes, Fullan and Quinn (2015) argue for external account- 
ability as part of strengthening school systems to deliver quality education for all 
children. 

The ALIVE initiative reveals that life skills can be measured, and can be mea- 
sured at household level. It is possible to assess the extent to which adolescents have 
mastered these skills in the contexts where they are expected to demonstrate them. 
This is timely feedback to the education system; one that is formally charged with 
the responsibility of nurturing life skills and values. In analysis of who should 
undertake an evaluation, Tyler (2005) argues that external parties should be the ones 
to undertake evaluation. The opportunity availed by HBA, such as expanding assess- 
ment of life skills and values to a wider population and aiding multi-level advocacy, 
cannot be overemphasised. Nonetheless, the approach also faces some limitations, 
including inability to reach all children in the target population as well as cost impli- 
cations caused by scattered samples, as distinct from centralised assessments where 
all those targeted are in one location (school). In the next section we describe some 
of these challenges. 


8.3.3 Challenges 


In this section, we reflect on the challenges that HBA face in assessing life skills, 
drawing on the ALIVE experience. The challenges relate to limitations of the 
approach in reaching all targeted children as well as administrative and cost impli- 
cations of the approach. 


Challenge One: HBA programs face the additional effort of defining their purpose 


Whereas the purpose and functions of school-based assessments are well known, 
there is no generally recognised purpose for HBA. The purpose and functions of an 
assessment program dictate the nature of the program, the form of the assessment, 
and how it is reported. Assessment is generally a core business of schools. Hence, 
when assessment takes place outside the school boundaries, it is often subjected to 
more scrutiny than school-based assessments. This explains the questions that have 
been occasionally raised against HBA (Nakabugo, 2021). In one instance, a Ugandan 
State Minister is quoted as slamming findings of an Uwezo report that had indicated 
that a significant number of children in upper primary classes lacked Primary 2 
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literacy and numeracy competences. He described the report as “malicious since the 
assessment parameters were not systematic and their credibility was questionable” 
(Kugonza, 2017, in Nakabugo, 2021, p. 53). To counteract such accusations, the 
assessment procedures must be rigorous and transparent. Uwezo ensures this, and it 
was also ensured in the context of assessing life skills and values in ALIVE. The 
processes of how the ALIVE assessment was conducted are well documented from 
setting the scope, task development, through to when the final tools were finalised 
and used for data collection. 


Challenge Two: Identification of target groups to assess in HBA programs may not be 
straightforward 


While assessment within the formal education sector typically targets students’ per- 
formance by grades and subject levels, and selection of the students is therefore 
straightforward, HBA requires intentional and systematic sampling strategies to be 
implemented in order to meet program goals. Sampling for HBA requires more 
elaborate processes. In the context of ALIVE, systematic steps were implemented to 
reach the targets set in the sampling frame. 

Clearly sampling for HBA is more elaborate than for those assessments con- 
ducted at school level. Hence, HBA may be more resource intensive than school- 
based assessments, a point elaborated below. 


Challenge Three: The processes, logistics planning and cost of HBA can be complex and 
intensive 


All assessment programs, whether school-based or community-based involve a 
series of pre-assessment, assessment and post-assessment activities, including 
mobilising people to undertake the different activities, introducing the assessment 
to different authorities, preparing related materials and tools, carrying out the 
assessment, monitoring and quality assuring the assessment processes, and receiv- 
ing the field returns and checking them for completeness. In addition, however, 
HBA requires intensive engagement due to the different layers of stakeholders 
involved and the fact that the assessment is conducted door to door, unlike school- 
based assessments where children to be assessed are all in one place. In the case of 
ALIVE, multiple players were involved. They included the core team that convened 
the expertise and the assessment infrastructure, the technical team that contextual- 
ised the assessment tool through a co-creation processes, government delivery sys- 
tems that were involved in the approval processes, and partners who coordinated the 
assessment, obtained approvals and consent at all levels and undertook the actual 
assessment. Costs associated with this process are due to the multiplicity of stake- 
holders and to the fact that a long-term infrastructure, such as exists in ministries of 
education, is not in place. 


Challenge Four: The choice of timing for conducting HBA is delicate 


A major challenge to HBA is to reach its target population. Unlike school-based 
assessment where the target group is already ‘captured’ and its availability known, 
this situation does not obtain in HBA. HBA must identify timing when likelihood of 
reaching the required samples (response rates) is maximised. HBA must be planned 
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and undertaken when the target groups are likely to be in the household. Therefore, 
assessments that target school-age individuals should be conducted after school and 
work hours (mostly evenings, weekends, and during school holidays). Since tar- 
geted respondents may well be temporarily out of the households when visits are 
made, the sampling strategy must include over-sampling estimates. Since likelihood 
of reaching target groups may vary, it is also necessary to undertake pre-assessment 
research to identify likely factors that might impact on availability. 

ALiVE was conducted during the school holidays. As a regional initiative, 
ALiVE was alive to the fact that the school calendars across the three countries vary. 
As such, it was impossible to standardise the timing of the data collection even after 
the completion of the logistics related to the assessments were completed. Although 
the sampling design included oversampling to deal with non-response and attrition 
cases, ALIVE stretched the data collection process allowing for callbacks to house- 
holds where adolescents were temporarily absent. In cases where listed households 
were completely unavailable, such households were replaced. 


Challenge Five: HBA may not reach all children after all 


HBA is presumed to have the power to obtain critical information about learning 
from all young people, not just those who attend school. Country-wide data on citi- 
zens is required, not just data on those who are ‘easy to access’, for country plan- 
ning across social, educational and economic sectors. Yet HBA also exclude some 
populations such as children in foster homes, street children and those living in 
exclusive institutions such as refugee settlements and barracks (Nakabugo, 2021). 
Among the out-of-school in the home, there may be children who are living with 
sensory impairments of hearing and vision whose functional disabilities preclude 
them from fully participating in the assessments. The cost of adapting the tools and 
expertise needed to avail the assessments to all children in the household, including 
those with severe disability, was not possible for ALIVE; children living with sen- 
sory disabilities were excluded from the study. 

In summary, HBA provide many opportunities to measure complex competen- 
cies such as life skills and values. Due to their broad target scope, HBA provide 
access to those who are not accessible through formal institutions such as schools. 
Despite the highlighted challenges of HBA, such as the populations they exclude 
and their resources and cost intensiveness, the network and collaboration of a wide 
range of stakeholders involved in their co-creation and execution makes them 
worthwhile. Most important to note is that life skills themselves are nurtured and 
demonstrated in spaces beyond schools such as households and communities. 
Explicit acknowledgement of these through the assessment initiative creates aware- 
ness of the importance and role of these skills and deepens conversations between 
citizen assessors and households. 


8 Opportunities and Challenges in Household-Based Assessment of Life Skills 131 


8.4 Conclusion 


The application of HBA in measuring education outcomes, especially life skills and 
values, is a relatively new idea. To date, influence of HBA, when applied to founda- 
tional skills of literacy and numeracy, on education policy and practice cannot be 
underestimated (World Bank, 2018). It has formed the basis for the World Bank 
framing the learning poverty phenomenon. In East Africa, program interventions 
have been designed and implemented in response to the body of knowledge gener- 
ated from the Uwezo HBA. The Education Programme for Results has been imple- 
mented in Tanzania since 2014, targeting children attending public schools that 
constitute the majority of the children missing out on foundational learning compe- 
tencies. Similarly, in Kenya, Primary Education Development Project, operational 
since 2016 and funded by the Global Partnership for Education, focused on improv- 
ing the teaching and learning of mathematics in Primary Grades 1-2 in Kenya (Piper 
et al., 2017). 

The process of applying HBA to life skills in ALIVE has revealed its immense 
capacity, and can seed consideration of assessment of other characteristics of inter- 
est. The approach can be replicated by assembling the right stakeholders to co- 
create the process. It is also clear that the process is not devoid of challenges. What 
is clear from the ALIVE process is that these challenges do not outweigh the oppor- 
tunities that HBA present for accessing information across the population. The 
documentation of process by ALIVE provides a valuable resource that can inform 
refinement, uptake and replication in other contexts. 

Finally, ALiVE has demonstrated that despite the challenges associated with 
assessing complex skills, it is possible to overcome them. What emerges is that 
HBA are agile and flexible to accommodate the measurement of skills considered 
complex such as life skills. By working with citizen surveyors and applying a col- 
laborative contextualisation of the assessments, the policy intentions can be achieved 
and advocacy for life skills made possible. 
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Chapter 9 A 
Scenario-Based Assessments: Experience as 
from East Africa 


Samuel Mutweleli ©, Grace Mwathe, and Scolastica Mundi 


Abstract How to assess twenty-first century skills continues to challenge measure- 
ment experts in education. Scenario-based assessment, a strategy used in the 
Assessment of Lifeskills and Values in East Africa (ALIVE) project, offers some 
benefits for these performance-based skills. In this chapter, scenario-based assess- 
ment is defined and described together with a discussion of assumptions made in 
developing such assessments. The creation of tools for assessing three life skills and 
the value of respect is used to illustrate issues encountered in the process. The daily 
life activities of adolescents in home, school and community environments in East 
Africa provided the substance for stimulus scenarios to which adolescents responded 
in this large-scale assessment program of more than 45,000 adolescents. The techni- 
cal processes undertaken by the test development team are described across stages 
of idea creation, think aloud, and dry run; culminating in a pilot in Kenya in order 
to generate data for reviewing scale development before the large-scale assessment. 
Through this process, lessons were learnt that may inform future use of scenario- 
based assessments in contexts similar to those in which ALIVE operated. 


9.1 Introduction 


In Kenya, Tanzania, and Uganda, education is undergoing reform in order to include 
values and competencies as integral components of learning, going beyond knowl- 
edge acquisition and emphasis on literacy and numeracy as the only foundational 
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skills (Kenya Institute of Curriculum Development, 2017). The inclusion of compe- 
tencies and values as part of the learning process broadens and challenges the form 
and use of assessment as these newly adopted competencies may require different 
assessment mechanisms. Educational assessment of traditional academic subjects 
has tended to focus on whether a competency or knowledge area has been achieved 
or not, with an assumption that there is a finite range which can be labelled as failure 
at one end, and success at another. A different approach is increasingly being applied 
to less content-focused learning. In this approach, there is no absolute range of per- 
formance, since what is being assessed are naturally occurring competencies. In 
addition, display of the competencies is typically viewed as context specific, being 
sensitive to critical features of the environment and the socio-cultural context. 

Several factors influence how assessments are designed. These include the pur- 
pose, the context, and practical constraints. Assessment tasks should provide individu- 
als with opportunities to demonstrate their competencies in meaningful contexts. 
Therefore, development of assessment tasks needs to be undertaken in such a way that 
the tasks target the intended population in a known context, and for a known purpose. 
Assessment tasks can take many forms, and selection of form is a key decision in this 
targeting. One form is the scenario, the subject of this chapter. Use of scenario-based 
assessment (SBA) has been viewed as effective in assessment of values, which are 
dependent on one’s perception of situations and learning experiences (Haynes et al., 
2009). In the case of values, these are both complex and formed by an array of social 
influences. Conventional test items that assume closed responses or closed self-report 
options may under-represent the complexity of some constructs. One of the features 
of SBA is that it can allow those assessed the opportunity to express themselves. In 
literacy assessment, for example, Sabatini et al. (2019) argue that there is need to 
move beyond the constraints of the traditional passage and question format of reading 
comprehension tests, and instead use scenarios to allow individuals the opportunities 
for self-expression. Similarly, for values, a method of assessment that allows adoles- 
cents to express themselves in their own words relative to specific scenarios and to 
consider their local context would be appropriate. 


9.1.1 Characteristics of Scenario-Based Assessment 


Scenario-based assessment is a method of evaluation in which situations that are 
hypothetical, but based in real life, are created, and to which individuals are asked 
to respond. An SBA task and its items typically takes the form of a brief description 
of a situation, followed by a series of questions or prompts designed to elicit behav- 
iours or perceptions associated with the target construct. SBA therefore provides a 
method for assessing individuals in a meaningful and purposeful context with tasks 
that relate to their experiences. The approach is useful for bridging across informal 
but complex social activities in order to provide information about human behaviour 
in real world situations (Haynes et al., 2009), in such a way that the formality and 
rigour required to generate robust and credible data is ensured. 
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Three characteristics of SBA are of particular interest in the case of assessment 
of life skills and values at household level, as envisaged by the Assessment of 
Lifeskills and Values in East Africa (ALIVE) initiative. These characteristics are 
familiarity, explanatory facility, and capture of complex phenomena. These charac- 
teristics are discussed in the context of the SBA design allowing for open-ended 
responses which are then coded according to criteria set to differentiate between 
performance levels. 


9.1.1.1 Familiarity 


The familiarity facility that SBA can offer is two-dimensional. First, the content of 
the scenarios can draw from daily life of the target respondents. Second, the admin- 
istration mode of SBA can be less formal such that it allays respondents’ assessment 
anxieties. SBA is apt for assessing competencies and values since these are acquired 
and developed through daily experiences - whether formally through structured 
learning processes, or informally as when adolescents interact in the community 
more broadly. Such experiences present the content used to stimulate responses 
(Haynes et al., 2009). Sabatini et al. (2019) argue that texts or stories used to assess 
literacy should reflect matters familiar to the target group. While Sabatini et al. 
focused on assessment of literacy, the lessons learnt can be extended to the assess- 
ment of life skills and values. 

The second dimension concerns administration mode. SBA’s use of an open- 
ended response format can reduce the perception that there is only one correct 
answer to an assessment item. This can encourage respondents to be more discur- 
sive in ways that reveal more about their perceptions and capacities. Scenarios can 
allow individuals to take varied approaches to demonstrating their capacities, mak- 
ing this approach beneficial for those who have difficulty with other assessment 
formats. The open-ended response mode can simulate a daily conversation in the 
hands of an experienced test administrator or interviewer. 

Life skills and values are attributes that relate to or direct one’s behaviour. They 
are presumed influenced by socialisation processes within the family and commu- 
nity. As such, the assessment of such attributes requires assessment approaches that 
reflect that socialisation. People generally find scenarios easy to identify and dis- 
cuss, as they constitute the patterns of their everyday life. The use of scenarios 
within the mode of less formal discussion is effective because the technique links 
what might otherwise just be stated, to actual activity or behaviour. This character- 
istic leads to the potential role of SBA as an explanatory facility. 


9.1.1.2 Explanatory Facility 
Using real-world problems and social context provides a natural base for assessment 


of cognitive abilities and motivation to participate in creative activities (Quitadamo & 
Brown, 2001). Twenty-first century skills, although diverse in nature, typically draw 
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on the analytical mind, on perspective taking, creative thinking, and flexibility. A well- 
constructed scenario triggers questions like, “What’s the goal of all this?” “What do I 
need to consider in this situation?” The scenario is ideally presented in such a way that 
respondents will think and react as naturally as they would in a real-life situation. 
Such assessments help learners to visualize, conceptualise, and comprehend the situ- 
ations presented. Accordingly, SBA can support learning. 

The use of real-life situations can validate individuals’ understanding of those 
situations and build their confidence in how to handle similar situations in future. 
One of the major benefits of using a scenario-based approach to learning is the 
development of the learner’s ability to comprehend situations and identify relevant 
factors (Bell et al., 2004). Learners can immerse themselves in a situation and prac- 
tice skills, while continuously learning from their mistakes. Driscoll and Carliner 
(2005) supported SBA due to the perspective that it enriches the process of learning 
by engaging learners in brainstorming, critical thinking, application and synthesis 
of real-life social situations. Use of relatable and engaging story-like scenarios reso- 
nates with learners. The experiences stick with learners longer, hence enhancing 
learning. The principles of situated learning theory (Lave & Wenger, 1991) argue 
that authentic activity, context, and culture are important components of knowledge 
acquisition. This takes us to the capacity of SBA to make visible the nature of the 
target construct, by virtue of constructing tasks and designing items that generate 
open-ended responses that are directly interpretable within the conceptual structure 
of the target construct. Haynes et al. (2009) in designing assessments for learning 
experiences, argue that scenario-based evaluation enables structured analysis of the 
causal forces behind phenomena of interest. 


9.1.1.3 Capture of Complex Phenomena 


Development of SBA tools assumes several features. First, the scenario will be able 
to address multiple aspects of the construct being targeted. These multiple aspects 
are defined by the hypothesised structure of the construct. ALIVE took a ‘construct- 
centred’ approach to its assessment design (Pellegrino et al., 2016). Second, the 
scenario must have qualities that enable different levels of performance to be elic- 
ited from the respondent. Third, and most critical, the questions that the scenario 
poses must be able to elicit participant responses that are clearly associated with the 
competence or value being assessed. In terms of the administrability of SBA, sce- 
narios must be written in such a way that respondents can reasonably be assumed to 
possess the requisite listening or reading comprehension in the language necessary 
to understand them. 

Construction of SBA tasks that adopt open-ended responses bypasses the need to 
develop series of branching pathways of prompts in order to capture the progress of 
individuals through multiple processes or steps, as is typically the mode of thinking 
through activities such as problem solving (e.g., Graesser et al., 2018). Instead, the 
design takes account of the key indicators of the steps, monitoring these, and using 
responses as indicative behaviours of interim processes. 
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Education systems have traditionally acknowledged acquisition of knowledge 
through tests and examinations practices which reward correct answers. The shift to 
twenty-first century skills which prioritises development of skills as described along 
learning continua requires assessment approaches that can capture increasing devel- 
opment. Such development is typically indicated by how an individual behaves or 
can perform, as distinct from what an individual might know. SBA is viewed as a 
reliable and valid assessment method for such performance-based constructs (Care 
& Kim, 2018). 


9.1.2 Use of Scenario-Based Assessment in ALIVE 


Reflecting the contextual philosophy of ALiVE, the project selected SBA as the 
predominant method for assessment of self-awareness and problem solving as well 
as the value of respect. The medium provided the opportunity to use local stories, 
narratives or situations to which adolescents could respond from within the house- 
hold and community environment. 

Each assessment task took the form of a hypothetical situation which was 
described orally to the respondent. The scenarios were based on real life contexts 
since the strategy was to use daily situations that would be familiar to respondents 
and would not assume knowledge based on formal education experience. After 
description of the scenario, a series of questions were asked about how the respon- 
dent might react to the situation described. The scenarios and their associated items 
(questions to the respondents) were designed to stimulate responses that would be 
interpretable within the conceptualisation of each of the constructs, and that could 
be construed to indicate proficiency level. 

The construction of the scenarios was therefore based on the hypothesised 
structures of each life skill and value, identified by dimensions and/or subskills 
(Scoular & Otieno, 2024; Chap. 6, this volume; Ngina et al., 2024; Chap. 5, this 
volume; Care & Giacomazzi, 2024; Chap. 4, this volume). Associated assess- 
ment frameworks were developed which included performance indicators. 
These frameworks guided the development of scenarios for each construct. In 
creating scenario ideas, the adolescents’ immediate environments were consid- 
ered, emanating in use of home, community, and school situations. 

Table 9.1 presents an example of a scenario-based task from the ALIVE tool. 
Given widespread familiarity with the nature of problem solving, this skill is used 
to illustrate the approach. The adopted structure for problem solving identified 
eight subskills which fell within three dimensions. The assessment framework 
operationalised four of these subskills organised within two of the dimensions 
(Care & Giacomazzi, 2024; Chap. 4, this volume). Decisions about the scope of 
assessment frameworks were made in recognition of the nature and limitations of 
the planned mode of assessment and its administration. The pragmatism of this 
approach is demonstrated in this example, where “applying the solution’ which is 
the third dimension of the adopted structure, is not included since it would not 
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Table 9.1 Illustrative task identifying subskills targeted and associated scoring rubrics 


Your parents left the house for work. You send your seven-year old sister to buy salt from the 
nearby shop. After 5 hours, your younger sister is not yet back. Your sister does not usually take 


all that time. 


Item Sub skill Scoring rubric 
Low Mid High 
Can you explain Recognizing | Irrelevant Relevant Relevant 
how this is a the problem | explanation explanation(s) explanations across 
problem? OR can within one two or more 
you explain how category: categories: 
this is not a Harm to the sister | Harm to the 
problem?) from accident sister from 
Harm to the sister | accidents 
from someone Harm to the 
else sister from 
Random event someone else 
Random event 
If you are asked to | Information | Response(s) not | Relevant Relevant responses 
solve this problem, | gathering related to the response(s) in one | across two 
what else do you problem category: categories: 
need to know about Information to Information to 
it? understand the understand the 
cause cause 
Information to Information to 
understand the understand the 
consequences consequences 
Suggest some ways | Exploring Response(s) not | Relevant Relevant responses 
to solve this alternative related to response(s) within | across two 
problem? solutions solving the one category: categories: 
problem Asking others for Asking others for 
help/ support help/support 
Searching for her Searching for her 
by self by self 
Of the suggested *Selecting Irrelevant Relevant response | Relevant response 
ways of solving the solution | response/ consistent with consistent with 


this problem, what 
is the best and 
why? 


response not 
consistent with 
previous item 


previous item, but 
with no or irrelevant 
justification 


previous item, and 
with relevant 
justification 


Note. * denotes that this item is not scored for respondents who do not provide a minimum of two 
solutions in the previous item 


have been viable within the timeframe and realities of the assessment event. To 
the degree that some dimensions and subskills were not measurable in ALIVE, it 
is essential that reporting of results clarifies the detail of what has and has not 
been measured. Examination of Table 9.1 shows how the phrasing of each ques- 
tion (item) targets the specific subskill which it is designed to stimulate. The SBA 
approach is totally dependent on the careful deconstruction of complex skills into 
their component parts. 
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9.2 Development of Scenario-Based Tasks 


A large technical team was formed to develop the ALiVE assessment tool. The team 
was composed of members from Kenya, Tanzania, and Uganda, and sub-divided 
into country-based groups for much of the development work. The full technical 
team participated in five workshops which focused on different aspects of test 
development. The aspects included but were not limited to: defining and describing 
the targeted skills and value; setting the scope and drafting the assessment frame- 
works; idea generation; and scenario development accompanied by scoring deci- 
sions. These tasks were undertaken through iterative processes of review, ‘think 
aloud’ and paneling activities, and piloting. Finally, data from the pilot were anal- 
ysed for final revision of materials. 
Those processes particular to task development are highlighted in this section. 


9.2.1 Idea Generation 


The process of idea generation entailed the consideration of the context of adoles- 
cents based on their daily life experiences with reference to family, school and com- 
munity activities and the probable roles of the youth in different tasks. It was 
challenging to generate ideas that would ‘carry’ content and prompts that would 
stimulate adolescents’ demonstration of competencies. The team needed to consider 
how familiar situations presented the opportunity for targeting the constructs and 
their subskills, having but recently gone through the process of deconstructing the 
skills. In addition, ideas were subject to debating cultural differences across the 
three countries, supported by the results of an earlier contextualization study 
(Giacomazzi, 2024). 


9.2.2 Checking Utility and Developing Rubrics 


The ‘think aloud’ activity was used to check first drafts of tasks. The goal was to 
determine whether an assessment task actually captured the intended competencies 
by analyzing the behaviours and metacognitive reflection of the responding adoles- 
cents as they worked through the tasks. The activity was also used as a strategy to 
collect likely responses which would inform the development of scoring criteria, or 
rubrics. “Think aloud’ involves individuals articulating their thinking as they com- 
plete a task (Eccles & Arsal, 2017). As adolescents went through each of the tasks, 
they were requested to report on their mental processes orally, explaining their 
thinking and reasoning. The transcripts of these think aloud activities informed the 
review process. Checklists were designed to identify whether: scenario tasks were 
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capturing the targeted competences, dimensions, and subskills; the tasks and items 
were clear; the tasks were perceived as familiar or reasonable by the adolescent. 

Where necessary the tools were translated to local languages used in Kenya, 
Tanzania, and Uganda. This was to eliminate any language comprehension barriers 
to actual understanding of the tasks. To harmonize the think aloud data collection 
across the three countries, each member of the National Technical Team participat- 
ing in the think aloud activity as an assessor was given standard field instructions 
which included a script to introduce oneself to the adolescent, clearly state one’s 
name, the organization represented, the purpose of the meeting with the adolescent, 
and what the adolescent was expected to do. All meetings with adolescents were 
undertaken by a pair of assessors. After the introductions, the two assessors mod- 
elled thinking aloud in order to help the adolescent understand what to do or how to 
respond to the items. Once the adolescent had fully understood what to do, the 
assessors read the tasks to the adolescent providing enough time for the adolescent 
to think through the task. In the event that the adolescent needed help with the item, 
the assessors repeated the item and/or asked the adolescent to think more. If the 
adolescent was not able to respond to the item, assessors would prompt to ascertain 
what kind of help was needed; for example, whether it was a comprehension, famil- 
iarity, or difficulty issue. This was recorded on the think aloud record form before 
moving to the next task. The record form included details on identity of assessors, 
the location where the activity was taking place, the adolescent’s name, sex, educa- 
tion (in school/out of school) and age. The record form had adequate space to allow 
the assessor to write for each item the adolescent’s responses, both initial responses 
and additional responses after probing. From each of the participating countries, 
three or four adolescents took part. 

A workshop was organized for country teams to present reports of the think 
aloud activity focusing on issues and solutions related to administration of the think 
aloud, usability of the tools, and behaviours of the adolescents as they responded to 
the tasks. In terms of administration issues, each team reported on: 


e Instructions to young person: what was the range of needs in terms of modelling 
what was required, or expanding on instructions? 

e Recording information: how adequate was the record form for capturing 
observations? 

e Physical environment for the process: were there any issues associated with the 
location for the activity that might impact on the young person’s performance or 
behaviour? 


The usability issues of interest were whether there was evidence that the adolescent 
found understanding instructions difficult or anomalous; whether the content of the 
tasks was perceived as sensitive in terms of culture, gender, religion or custom. The 
main question for the think aloud activity was whether the task and its items 
appeared to capture the targeted subskill. The question was answered through two 
processes: first was aggregating the response data from across the three countries’ 
think aloud activity, using the same protocols to ensure comparability across records 
and across country, and adding commentary based on observations of the 
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adolescents; second was analysing the data in order to make decisions about the 
utility of the tasks and items for targeting the skills and subskills and for capturing 
different levels of quality, and to initiate the development of scoring criteria. 


9.2.3 Paneling 


Armed with the information derived from the think aloud process, the technical 
team proceeded to review tasks to decide which would proceed to the pilot. Paneling 
is a quality assurance process to check and improve draft assessment items in terms 
of content and construct validity, and capacity of the items to capture differences in 
performance. A panel was formed to focus on each construct. The model for alloca- 
tion of members to panels was to allocate a minimum of two individuals who had 
worked on a construct to that construct’s panel in order to ensure expertise to 
respond to other panelists’ queries; and roughly equal allocations of individuals 
from other construct areas. The role of the two ‘construct representatives’ was to 
respond to queries, to explain and to clarify, not to defend or argue the case. 
Membership of the panels was: 14 for problem solving, 13 for self-awareness, and 
9 for respect. Each panel was guided by a paneling checklist in their review of draft 
tasks. The panels examined whether the task and item combinations assessed (part 
of) the construct, what respondents would need to know to respond to the scenarios, 
the authenticity of the scenarios, the precision and clarity of the phrasing of tasks 
and items, the amount of time needed to produce an answer, adequacy of the scoring 
rubrics, and equity for respondents of different backgrounds. A sample of items in 
the paneling checklist is provided in Table 9.2. 


Table 9.2 Sample items on the Panelling checklist 


Does the item test the underlying construct that is the focus? 


Is the item at the right level in terms of the expected ages of the test-takers? 


Is the form, presentation, and language of the item coherent? Unambiguous? Clear? 


Is the question expressed as briefly as possible? 


Are assumptions of prior knowledge in the item appropriate? 


If a constructed response item, is it clear what would constitute an answer to the question? 
That is, will test-takers know what they are being asked to produce? 


Are there dependencies between this item and others, or between parts within the item? 
Does one item or part give a clue to the next one? Is this intentional? 


Does the way the item or task is phrased, provide a clue to a preferred response? 


Is there reasonably a possibility that responses can be given over a range of proficiencies? 


How will the test-takers perceive this material? (group members should imagine the cognitive or 
social demands of the items from the point of view of test-takers) 


Is the item likely to be biased, that is, is it likely to be easier or more difficult for subgroups for 
reasons other than differences in the ability being measured? 


Does the item breach ethical, cultural or other sensitivities? 
How would the material stand up to public scrutiny? 
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The panels summarised the results of their deliberations and recommendations 
using a paneling summary form whereby members reported the quality of the task 
in terms of how effectively it captured the subskills targeted, and whether its items 
had potential for differentiation across different levels of proficiency. Members 
were then required to make a preliminary decision on whether a task or its item/s 
should be edited, discarded or retained. 

An important activity which took place at this stage was the determination of the 
levels of difficulty of the assessment tasks. The utility of tasks and items depends 
not only on whether they appear authentic and targeting the skills, but on whether 
they can differentiate between levels of proficiency. The issue of whether the tasks 
and items could identify levels of proficiency was explored using two sources of 
information: 


1. A priori descriptions of hypothesised indicative responses 
2. Analysis of responses from the think alouds to identify different levels. 


The activity informed reciprocal goals: identifying if the scenarios could stimulate 
responses across a range of lower to higher proficiencies; and developing rubrics 
that could describe these levels of proficiency based on adolescents’ responses. 
Each ‘skills team’ compared the think aloud data with a priori descriptions of the 
hypothesised responses or behaviours. The teams checked frequency data of 
responses collected through the think aloud activity against the a priori responses 
ranging from low to high proficiency levels. Where there were no frequencies 
against a level, this stimulated discussion on whether to improve or drop the task. 
Another method used involved comparison of think aloud-derived responses with 
each other. Teams transcribed the individual responses onto sticky-notes and 
arranged these in ascending or descending order according to members’ judgement 
of increasing or decreasing proficiency. The final step was the development of 
rubrics that distinguished between proficiency levels. If rubrics that could compre- 
hend the responses for coding could not be established, the task was discarded. This 
establishment of coding criteria, or rubrics, was a pre-requisite for the next activity, 
the pilot study. 


9.2.4 Pilot and Dry Run Studies 


The pilot study was undertaken across Kenya, Tanzania, and Uganda in November 
2021. The number of adolescents participating varied from N = 366 for respect, to 
N = 392 for self-awareness and N = 395 for problem solving. Guidelines for the 
exercise covered what the assessors were expected to do before, during, and after 
the assessment with regard to safety precautions, protocols to be followed, and ethi- 
cal considerations. The pilot generated identification of terminology and language 
issues; and response data from the adolescents, both qualitative and quantitative. 
Data analysis involved calculation of descriptive statistics, including frequency dis- 
tributions, measures of central tendency and dispersion. Item response data were 
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used to explore scale composition, followed by establishing model fit. The output of 
these analyses, supplemented by analyses of dry run data, was used to make final 
decisions on each SBA task to be used in the large-scale assessment. Notably, the 
pilot collected not only data coded according to the [then] rubrics, but also the ver- 
batim responses of adolescents. This rich data source provided the facility for fur- 
ther review of coding categories where anomalous patterns of data were apparent. 
In turn, this informed slight changes in scenario wording and rubrics criteria prior 
to the dry run. 

A dry run was undertaken with N = 337 adolescents in Kenya in February 2022 
as a test of the whole process, as distinct from the earlier pilot for which the focus 
was on finalization of tools based on how the test items functioned. The dry run 
provided field experience to guide the fine-tuning of administration guidelines and 
some slight re-phrasing of scenarios. The dry run used cell-phone based data cap- 
ture through KoboCollect, and only coded responses to items were recorded. An 
analysis team comprised of 13 members from the original technical team as well as 
academics and statisticians drawn from universities and the government sector 
engaged in exploration of the dry run data. Again, item response distributions were 
analysed, scales were reviewed, and associations between the constructs explored. 
These activities led the team to reach: 


e Consensus on the underlying definitions and descriptions of the constructs (prob- 
lem solving, self-awareness, and respect) 

e Agreement on the skills structures of the target competencies. 

e Agreement on the coding of the target competences. 


That ALIVE decided to adopt SBA as the main assessment approach is not a claim 
that the method is perfect, either for the target constructs, or for the contexts in 
which the assessments were to take place. Challenges remain. 


9.3 Challenges of SBA Illustrated 


As with development and use of any assessment methods, what they offer in terms 
of benefits does not come easily. Some of the challenges associated with use of SBA 
are illustrated through ALiVE’s experience. 

Assessment of twenty-first century skills demands multiple modes or multiple 
data points in order to capture their complexity effectively. Since many of these 
skills are demonstrated through behaviour rather than through written or oral 
responses, observation of behaviour as individuals perform tasks would be recom- 
mended, all things being equal. The SBA approach taken by ALiVE opts for imme- 
diate judgment by assessors concerning the relevance and skill level of an individual 
as they perform or provide a response. There is therefore an inherent risk posed by 
the capacity of assessors to capture and interpret behaviour accurately. According to 
Pellegrino et al. (2016), an assessment is designed to observe learners’ behaviour 
and draw inferences about what the learner knows or can do. The detailed 
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development processes undertaken in this study were designed to identify fit-for- 
purpose coding criteria that would be relatively easy to understand and apply during 
an assessment event in the field. From the robust scales derived from the large-scale 
study results (Ariapa et al., 2024), it is clear that such understanding and application 
was possible in this instance. However, if the SBA offered stimulus for a wider 
range of proficiencies, coding would arguably present greater challenges. Hence the 
approach has a clear limitation for application in contexts such as household-based 
assessment. 

Whitlock and Nanavati (2013) argue that acquisition and demonstration of skills 
in assessment contexts may not strongly relate to the skills that are brought to bear 
in real-world experience. This is precisely the situation that SBA avoids since, in 
this current application, it is to familiar situations that adolescents are asked to 
respond. Of course, a response to a hypothetical, even if authentic, situation is not 
the same as a true response triggered by events. To this extent, SBA cannot emulate 
real life. In addition, the need for simplicity of situation to act as a stimulus might 
under-represent the real demand of such situations and so be incapable of stimulat- 
ing less visible skills that might be part of a natural response. What is important is 
that the SBA has the capacity to stimulate different responses across different indi- 
viduals. This was witnessed in the ALIVE assessments as adolescents interpreted 
and responded to similar tasks differently. For instance, the first scenario on prob- 
lem solving focused on fire breaking out and the adolescents were required to 
respond to the first task asking whether they considered this situation to be a prob- 
lem. A sample of responses showing different levels of proficiencies is given in 
Table 9.3. 

Putting aside where such differences in response are accounted for by variations 
in proficiency, the assumption that responses are fully determined by the construct 


Table 9.3 Verbatim responses representing different proficiencies 


Response Proficiency level 
0 1 2 3 
No response/| Irrelevant | Relevant explanation(s) | Relevant and multiple 
I don’t know | explanation within one category explanations across 
(a) damage to property; both categories 
(b) risk to life (a) damage to property; 
(b) risk to life 
No response 0 
I’m not the 1 
one who 
started the fire 
Yes, because 2 
the house is 
on fire 
Yes, it 3 
destroys 
property and 
people may 
die 
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being targeted, is questionable. One of the significant challenges in assessment of 
complex social skills is that these are influenced by multiple phenomena, some of 
which are cognitive, some interpersonal, some intrapersonal, and some contextual. 
Ascribing a response or reaction to just one source, whether it be a trait or a situa- 
tional state, is to ignore the complexity of human being and behaviour. This remains 
a challenge in SBA and for ALIVE. As with other methods of assessment, the claims 
for construct validity will remain for adjudication by replication. 

The development of SBA entails investment of time and effort to develop rele- 
vant and engaging scenarios to assess the intended construct, and according to Ryan 
(2006) may not necessarily provide the intended value. One challenge is designing 
an administrable assessment that can encompass all the intended elements of the 
target construct. The SBA development team needed to keep in mind that the goal 
was to sample adolescent proficiencies for the purpose of establishing regional pro- 
files, rather than to comprehensively describe those proficiencies at individual level. 
In addition, the household-based assessment context would be such that maintain- 
ing adolescent attention beyond 40-50 min would be unsustainable. These realities 
led to the diminution of the original construct structures to the assessment frame- 
works — the more easily assessable elements would be targeted in the interest of 
evaluating these reliably. 

It is worth noting that SBA, as a method and as an interaction mode was in general 
not familiar to the adolescents. Hence, they may have experienced some confusion 
about what was expected of them, notwithstanding the overwhelmingly positive moti- 
vation to be involved in the activity. A general challenge of SBA may have exacer- 
bated that confusion: the ability of individuals to perform tasks within constrained 
time limits due to task complexity. Building mental models is known to be complex 
and individuals may take time to navigate the complexities of extracting meaning and 
integrating background knowledge (Kintsch, 2012). Whether this in fact did impact 
on adolescent capacity to demonstrate optimal levels of skill is not known. 


9.4 Conclusion 


The use of SBA in the ALIVE project served a major requirement that the assessment 
of life skills and a value be contextualised in terms of the constructs themselves, how 
they play a role in daily life, and where they are demonstrated. These three conditions 
established a context in which adolescents were presumed to enter the assessment 
event confidently. The scenarios themselves were key in confirming familiarity of 
content in the actual tasks. The contextualisation of the skills and value themselves 
in the early part of the study (Giacomazzi, 2024; Chap. 3, this volume) ensured that 
interpretation was such that tasks would stimulate the cognitive and social-emotional 
processes of interest. The place provided the actual physical environment in which 
the scenarios would typically play out. In addition, the simplicity of the scenarios 
used for the tasks was such that task descriptions and prompts could be translated and 
administered in the local languages of the adolescents. 
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Establishing these conditions does not however guarantee accuracy of results, 
particularly the proficiency levels estimated on the basis of adolescent responses. A 
question that remains to be answered is whether the results from the assessment 
under-estimate adolescents’ actual proficiencies. This possibility is raised due to the 
observation made by some assessors that adolescents were unaccustomed to the 
assessment mode and unsure of how to respond. The mode was less formal than 
routinely associated with assessment activity, and less formal than an adolescent 
might expect with an adult with whom they are unfamiliar. The training of assessors 
needs to be comprehensive in order to approach and interact with the adolescent 
appropriate to the goal of the activity, and to master the coding rubrics. If assessors 
are not properly trained, they are likely introducing measurement error by either 
under or over reporting the proficiency levels. The practice of pairing of assessors to 
administer the tasks and code the adolescents’ responses minimised the coding 
errors hence enhancing the reliability and validity of the findings. 
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Chapter 10 A 
Measuring Adolescents’ Life Skills gues 
and Values: Method and Results from East 
Africa 


Martin Ariapa ©, Masa Pavlovic ®©, and Esther Care © 


Abstract The goals of the ALIVE initiative, to capture and disseminate estimates 
of adolescents’ life skills and values at a level that would provide valuable informa- 
tion to ministries of education, guided the development of an assessment tool. The 
aim was to produce a tool that had the capacity to assess the proficiencies of adoles- 
cents in East Africa through a household-based approach. This approach required 
that the tool could capture these cognitive, interpersonal, and intrapersonal capaci- 
ties; not rely on knowledge and skills built through formal education; and would be 
compatible with oral administration of the assessment with participating adoles- 
cents. The process of development is described as taking place within these param- 
eters. Results from the assessment of 45,442 adolescents across Kenya, Tanzania, 
and Uganda are presented following the technical information generated through 
the test and scale development process. The patterns of results are uniformly similar 
across each of the education systems, although there are some slight differences in 
levels of proficiencies. The outcomes provide confidence in the test and scale devel- 
opment processes followed, and in the use of household-based assessment for mea- 
surement of life skills and values. 


10.1 Introduction 


This chapter provides two main sets of information. First, it describes the develop- 
ment and validation of the tool designed to measure proficiencies demonstrated by 
adolescents aged 13-17 years in Kenya, Tanzania, and Uganda in the three life skills 
and one value measured under the Assessment of Life skills and Values in East 
Africa (ALIVE) initiative. Second, it provides and discusses results from the 
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large-scale assessment program that relied on the tool. The information provides the 
background to the assessment results that is needed to inform policy of the partici- 
pating countries as they seek to include life skills and values in their national 
curricula. 

ALiVE began in August 2020, starting with a contextualisation study designed 
to explore the nature of life skills and values in the East African community. With 
outputs from this study, a process of re-framing these ‘constructs’ (a term that is 
used to represent particular constellations or groups of behaviours or characteris- 
tics) was undertaken to ensure appropriate interpretation in the East African com- 
munity. The activity reflected a well-recognised model of creating psycho-social 
measurements (Wilson, 2005) in which the process requires clarification of the 
construct, development of tasks and items, collection and scoring of responses, 
and production of measures. From the contextualisation study, the conceptual 
structures of the target constructs were agreed upon, followed by development of 
assessment frameworks which identified the measurable components of those 
constructs. Guided by the assessment frameworks, the ALiVE team moved into 
task and item development in March 2021, which was finalised by August 2022. 
The large-scale assessments across the three countries were administered to over 
45,000 adolescents late in 2022. 

Development of tasks and items to measure the constructs was undertaken 
mindful of the intended mode of data collection and of the intended use of results. 
Of particular note was that results were to be used to provide broad descriptors of 
adolescent proficiencies at aggregate levels. There was no assumption that results 
could or would be used to describe the proficiencies at individual level or for 
diagnostic purposes. Accordingly, the test and scale development process was 
focussed on identification of key indicators of the constructs and their subskills 
rather than a comprehensive and deep evaluation of demonstration of competen- 
cies. The goal guiding the process was to enable reporting of competencies in a 
way that would provide ministries of education with sufficient information upon 
which to consider best integration of these into mainstream curricula. As docu- 
mented in Care (2024; Chap. 1, this volume) all three participating countries and 
their four education systems have either recently incorporated these competencies 
into their curricula or are in the process of doing so. Several are in the process of 
delineating how, and at what levels, integration should be managed. The informa- 
tion generated by the ALiVE program therefore provides empirical evidence of 
current levels of functioning in the 13—17 year old age group that can be factored 
into decision-making by the national education systems. The constructs for which 
a tool was created are collaboration, problem solving, respect, and 
self-awareness. 
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10.2 Method 


10.2.1 Design 


The design of the tool was influenced by several factors. The first of these was the 
use to which the assessments would be put; the second was the intended mode of 
data collection; the third was principles of test and scale development. 


ALIVE was designed to capture a glimpse of functioning across life skills and 
values, as aspired to by ministries of education in the respective educational 
jurisdictions (Kenya, Tanzania mainland, Uganda, and Zanzibar). The assess- 
ments were not designed for diagnosis of individual functioning but rather to 
establish a basis upon which countries might evaluate their educational goals 
given their embrace of life skills and values in recent years, and to inform their 
curricular planning. 

ALIVE was interested in a representative sample of the participating countries’ 
adolescents who might be in or out of school or employment, since the targeted 
competencies have been highlighted by employers and in the technical-vocational 
sector as well as by ministries of education. This interest, therefore, required 
household-based assessment. This medium for assessment, in turn, requires 
manageable interactions in the field distinct from interactions that can be man- 
aged at the group level in a formal education environment. Manageability in the 
field implies assessment forms that can be communicated orally, in time-efficient 
ways, and through content such as daily life scenarios that are not reliant on 
school-based learning. 

ALiVE committed considerable effort to definition and description of the target 
constructs. This was undertaken both due to observance of a well-established test 
and scale development model (Wu et al., 2016), and to the combination of two 
relatively recent innovations. First, the assessment of twenty-first century com- 
petencies remains in its early days (Griffin et al., 2012; Care et al., 2018), and 
second, household based assessment at large scale has emerged in the past decade 
as an acceptable and sufficiently stringent approach to collection of data that 
prompts government action (e.g. Uwezo, 2021) or contributes to monitoring of 
the Sustainable Development Goals (UNESCO UIS, 2022). The household mode 
has been used predominantly for measurement of numeracy and literacy rather 
than competencies associated with socio-emotional functioning. Ensuring com- 
mon understanding of the target competencies is therefore not only essential in 
the routine test and scale development processes, but in ensuring same under- 
standings when hundreds of Test Administrators and other personnel are involved 
in data collection. 
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10.2.2 Sample 


The sampling frame used for this study was derived from the Population and 
Housing Census frames for Kenya, Tanzania (mainland and Zanzibar), and Uganda. 
This frame included a complete listing of census enumeration areas and households. 
In each country, a multi-stage sampling approach was used to select households and 
adolescents for the assessment. The approach involved selection of districts/coun- 
ties; followed by the selection of enumeration areas; that is, the smallest areas of 
clustering the population within a country; and then selection of households within 
each selected enumeration area. The desired sample sizes were determined by con- 
sidering: the degree of precision desired for the study estimates, the cost and opera- 
tional limitations, the efficiency of the design, and a fixed number of households per 
enumeration area. A total of 45,442 in-school and out-of-school adolescent boys 
and girls aged 13-17 years from 35,720 households, 1991 enumeration areas, and 
85 districts/counties were assessed as shown in Table 10.1. 

Of the enumeration areas, reached were just 18 less than the planned allocations. 
Approximately 10% less households were reached than planned (35,720 of 40,000). 
Discrepancies in both cases were distributed across each of the jurisdictions. 
Sampling weights calculated based on sampling probabilities for each sampling 
stage were used in the analyses to ensure that the results accurately reflect the char- 
acteristics of the population being studied. Adjustments for non-response were also 
made by including in the sampling weights the household response rate adjustment 
factor. The sampling weights were then used to adjust analysis of the sample data to 
account for any differences between the sample and the population. 


10.2.3 Ethical Considerations 


The research protocols were reviewed and approved by the relevant research regula- 
tory authorities in the four jurisdictions. Practical strategies adopted to safeguard 
participants’ rights and safety included obtaining informed consent from all partici- 
pants and, as far as possible, ensuring voluntary participation. Both parent or guard- 
ian consent and adolescents’ consent were sought before the start of the assessment. 
Individuals’ identity was safeguarded through codes in storing and sharing data. 


Table 10.1 Sampling across districts, enumeration areas, , and adolescents 


Variables Kenya | Tanzania Mainland | Uganda | Zanzibar | Total 
-Districts/counties 20 34 o n 85 

Enumeration areas 798 673 | 400 120 1991 

Households 14,161 11,802 | 7815 1942 35,720 

Adolescents (problem solving, 17,276 14,645 11,074 2447 45,442 

self-awareness and respect) 

Adolescents (collaboration) 7494 6827 4476 | 1319 | 20,116 
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The activities were undertaken in recognition of three risk factors that high- 
lighted the project as requiring of human research ethics attention. These included: 
the age of the young people was such that they were regarded as a vulnerable group; 
the area of enquiry included self-awareness and respect, two social functioning 
aspects of life that can be seen as sensitive; the scenario form of assessment was 
contextualised within issues that could occur in daily life, which could reasonably 
be seen as sensitive topics for some. 

Two approaches to risk minimisation were taken. First, the development of the 
tool included multiple iterations of review of content in order to ensure that this 
would not confront the participants with issues relating to sensitivities associated 
with culture, gender, ethnicity, language, or religion. Second, the training of Test 
Administrators included sensitivity awareness to such issues and provided clear 
guidelines for how to approach and interact with the participating adolescents and 
their parents. 


10.2.4 Data Collection 


The ALiVE assessment events were conducted by 3080 Test Administrators (TA) 
across the four jurisdictions—with two of these allocated for each adolescent inter- 
viewed. These assessors participated in three-day training workshops which intro- 
duced them to the purpose of the assessment, the nature of the targeted constructs, 
the assessment tool and scoring rubrics, and ethical issues and principles. They also 
participated in a field practice exercise to familiarise themselves with the assess- 
ment protocol. 
The data collection tools drew on two formats: 


e Scenario-based tasks were used to assess problem solving, self-awareness, and 
respect. The scenarios were based on elements of daily life with which adoles- 
cents could be expected to be familiar. Each scenario, of 1—4 sentences, was read 
aloud to the adolescent, followed by a series of questions. Either English or the 
local language was used. Adolescents responded orally, again in English or the 
local language, as preferred, with TA noting down the responses and coding 
these based on scoring rubrics. TA were paired so notations and coding could 
occur in real-time and be reviewed at the day’s end for verification before 
submission. The number of tasks and associated items are summarised in 
Table 10.2. 

e Performance-based tasks were used to assess collaboration. The tasks were 
administered to groups of four adolescents in both single-sex and mixed-sex 
groups. Adolescents who had already completed the individual scenario-based 
tasks of problem solving, respect and self-awareness, formed these groups. The 
tasks were read aloud to the group in the language preferred by adolescents. The 
first two tasks required the group to work with physical materials available within 
the immediate environment or provided to the groups. TA were again paired such 
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Table 10.2 Numbers of tasks and items in the ALIVE tool set 


Constructs # items 
Problem solving: 3 task scenarios 12 
Dimensions and Subskills Defining the problem 6 
Recognising the problem 3 
Information gathering 3 
Finding the solution 6 
Exploring alternative solutions 3 
Selecting the solution 3 
Self-awareness: 5 task scenarios 12 
Dimensions Internal self-awareness 6 
External self-awareness 6 
Respect: 4 task scenarios 10 
Dimensions | Regard for others 10 
Collaboration: 3 performance tasks 8 
Dimensions Communication 3 
Negotiation 3 
Working together 2 


that the notations and coding could occur in real time, and be reviewed at the 
day’s end for verification before submission. The number of tasks and associated 
items are summarised in Table 10.2. 

e A household survey tool was used to collect household and adolescent back- 
ground data. Information gathered included: household head (gender, educa- 
tion), number of members in the household, types of walls of the main house, 
main source of lighting, the main source of water, number of meals regularly 
eaten, number of assets owned, media preference and use, and adolescents’ char- 
acteristics such as gender, education level, age, and functioning (hearing, seeing, 
walking, remembering and self-care). The household tool was responded to by 
the head of the household. 


Recording of responses from all tools was enabled through the TA use of 
KoboToolbox on mobile devices (tablets or phones) (Kobo Organization, n.d.). 
Case records were uploaded at the end of each day of testing to the main data 
repository. 

Table 10.2 makes explicit both the target life skills and one value and their contrib- 
uting dimensions or subskills. Note that the dimensions and subskills do not fully 
represent the overall constructs in every case. Additional information about the con- 
ceptual structure and assessment framework for each construct is found in each of the 
three chapters dedicated to these (Scoular & Otieno, 2024; Chap. 6, this volume; 
Ngina et al., 2024; Chap. 5, this volume; Care & Giacomazzi, 2024; Chap. 4, this 
volume). As made explicit by these sources, the assessment frameworks which led the 
design of the tasks and items were guided by the conceptual models but determined 
by the pragmatics of the planned assessment approach. Responding to the household- 
based environment, the conditions for data capture were such that the more easily 
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identifiable indicators of the various constructs were prioritised for measurement. 
Tasks and items were developed to encompass the dimensions and subskills identified 
in Table 10.2. This chapter reports on the main construct for collaboration, problem 
solving, and respect, and across the two dimensions for self-awareness. 


10.2.5 Structure of the Tools 
10.2.5.1 Collaboration 


The assessment constitutes three tasks, with a subset of 8 items. Two tasks follow 
the same pattern, targeting a step approach to collaboration. The items assess the 
adolescent’s communication—which is about listening (receptive) and speaking 
(expressing); negotiation—which is needed for one to reflect on other people’s 
views vis 4 vis their own, including accepting feedback and reaching a consensus; 
and working together—to plan and engage in activities. The last task includes only 
the communication and negotiation phases. The final tool provides 8 data points. 
Additional information on the construct can be found in Scoular and Otieno (2024; 
Chap. 6, this volume). 


10.2.5.2 Problem Solving 


The assessment constitutes three contextualised task scenarios, each containing four 
items. A task scenario comprises a brief description of a situation with four items, 
each of which targets a different aspect of an adolescent’s problem-solving profi- 
ciency. All three task scenarios follow the same pattern, with each of the four items 
targeting a step or process approach to problem solving. The first two items of a task 
scenario assess the adolescent’s recognition of the problem, followed by gathering 
relevant information. The second two items assess the adolescent’s exploration of 
alternative solutions and selection of the best solution. The final tool therefore pro- 
vides 12 data points. Additional information on the construct can be found in Care 
and Giacomazzi (2024; Chap. 4, this volume). 


10.2.5.3 Respect 


The assessment constitutes four task scenarios with a subset of 10 items. All four 
task scenarios follow a similar pattern, from awareness of lack of kindness to recog- 
nition of actions betokening lack of respect. The items assess the adolescent’s regard 
for others and regard for property: awareness of infringing on others’ rights, recog- 
nition of one’s wrongdoing, respect for the rights of others, and willingness to make 
amends for wrongdoing. The final tool provides 10 data points. Additional informa- 
tion on the construct can be found in Ngina et al. (2024; Chap. 5, this volume). 
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10.2.5.4 Self-Awareness 


The assessment constitutes five task scenarios with a subset of 12 items. Unlike the 
case of the problem-solving tasks and items, the items across these five tasks follow 
slightly different patterns. The items assess the adolescent’s self-awareness through 
two subskills: se/f-management—managing emotions and stress; and perspective 
taking—understanding views and actions of others, adjusting to others’ views and 
actions, and recognising one’s identity and where one fits in family, society, and 
community. The final tool provides 12 data points. Additional information on the 
construct can be found in Ngina et al. (2024; Chap. 5, this volume). 


10.2.6 Data Analysis 


Initially, classical test theory was used to explore the functioning of tools across its 
items. Specifically reviewed were: the distribution of responses across items; the 
patterns of responses for each item by country, gender, age, and education levels of 
the adolescents. Reliability coefficients were then calculated to establish the scales’ 
coherence. 

The Rasch model was used to explore and quantify the participants’ 
responses.' Using the Rasch model provided tools for interpreting skills that 
underpin constructing and developing empirical proficiency levels. Proficiency 
levels describing increasing competency levels were developed for the over-arch- 
ing constructs. Following these test and scale development processes, further 


'The Rasch model was used to calibrate the participants’ response data. A Rasch analysis places 
the participants’ ability and item difficulty on a common metric. The Rasch model provides a 
method for scaling persons and items on the same metric, called a logit scale, which allows for 
direct comparisons between item/task difficulty and students’ ability. The model requirements are 
that item parameters are independent of the students/participant being measured. Similarly, esti- 
mates of participant abilities are independent of a set of items used to measure underlying ability/ 
trait. In order to use the model, it is necessary to test for the fit, which indicates how well the data 
meet the model requirement. Fit statistics are used to investigate how each data point (both at the 
item and person level) conforms to the expectations of the model. Misfitting items or persons can 
distort measurement and may suggest that some items do not align with the construct being mea- 
sured or that some persons did not engage with the items as expected. Examination of item fit and 
spread provided evidence for construct and criterion validity. 

Differential item functioning (DIF) was conducted to establish the scope of use and the gener- 
alisability of results across and within countries. DIF is a statistical method used to detect differ- 
ences in item performance for individuals with the same underlying ability but belonging to 
different subgroups. In the context of the psychometric analysis, the presence of DIF can indicate 
the presence of bias and impact. Bias refers to a systematic error that unfairly advantages or disad- 
vantages certain groups of test-takers. If an item is biased, it may be easier or harder for certain 
groups of assessed students, not because of differences in the test takers ability but because of other 
unrelated factors. On the other hand, an item may show DIF due to differences in educational 
opportunities or cultural experiences, which are not biased but rather real differences in ability 
distribution on the measured trait. That is, bias can lead to DIF, but not all DIF is due to bias. 
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analyses were performed to explore the four constructs across jurisdictions and by 
selected variables—gender, education level, adolescent age, and disability status. 

Finally, based on the Rasch model analysis outputs, patterns were recognised 
regarding increasing proficiency levels across the four main scales and the subscales 
for problem solving, self-awareness and collaboration. The results demonstrated the 
utility of the rubrics used for coding responses. 


10.3 Results 


10.3.1 Demographic Characteristics of Adolescents 


Younger and older adolescents were almost equally distributed across the two 
main age ranges (13—14 years and 15-17 years) (Table 10.3). Approximately 13% 
of the adolescents assessed were out of school—not currently studying. For ado- 
lescents not currently in school, the achieved highest level of education was 
recorded as primary or secondary for analysis. Across the full sample, about 30% 
of adolescents assessed reached secondary education level, and 65% reached pri- 
mary education level. Note that there are some variations across the four jurisdic- 
tions in allocation of Grades 7 and 8 to primary versus secondary education. 

Disability status of the adolescents was determined using the Washington Group 
Short Set of Questions. Parents were asked whether their children had any difficulty 
in vision, hearing, walking, memory, self-care and language/communication and 
how severe such difficulty was. Overall, parents reported about 12% of the adoles- 
cents had some form of difficulty. 


Table 10.3 Gender, age distribution and education status of adolescents 


Male Female Other Total 
n(%) n(%) n(%) n(%) 
Gender 22,092(48.6) 23,264(51.2) 86(0.2) 45,442(100.0) 
Age group 
13-14 years 10,699(48.4) 11,860(51.0) | 46(53.5) 22,605(49.7) 
15-17 years 11,393(51.6) 11,404(49.0) | 40(46.5) 22,837(50.3) 
Schooling status 
In school 18,842(85.3) 20,555(88.4) | 73(84.9) 39,470(86.9) 
Out of school 3250(14.7) 2709(11.6) 13(15.1) 5972(13.1) 
Education level 
Primary 14,956(67.7) 14,602(62.8) 53(61.6) 29,611(65.2) 
Secondary 6015(27.2) 7586(32.6) 28(32.6) 13,629(30.0) 
Disability status 
No form of disability 19,542(88.5) 20,407(87.7) | 75(87.2) 40,024(88.1) 
At least 1 form of disability 2550(11.5) 2857(12.3) 11(12.8) 5418(11.9) 


160 M. Ariapa et al. 


10.3.2 Psychometric Properties 
10.3.2.1 Reliability Analysis of the Tools 


Table 10.4 presents alpha coefficients as a measure of internal consistency in the 
scales for the over-arching constructs and relevant dimensions for collaboration, 
problem solving, respect, and self-awareness. 

For all scales, the alpha reliability coefficients indicate high homogeneity of con- 
tent. The average inter-item correlation coefficients for the over-arching constructs 
ranged from 0.300 (for self-awareness) to 0.448 (for problem solving), suggesting 
that the target items are reasonably homogeneous, and contribute unique variance to 
their over-arching constructs. 


10.3.2.2 Item Fit Statistics” 


All items across the four constructs demonstrated ‘good’ fit statistics. That is to say, 
the weighted mean-square values were all between 0.7 and 1.3 (Wu et al., 2016). 
See Appendix 1 (a, b, c, d) for the mean squared residual based item fit statistics, 
weighted and unweighted. 


10.3.2.3 Differential Item Functioning 


Use of assessment tools across countries or cultures raises issues of validity of com- 
parisons between groups. Such issues may reside in matters of language, societal 
norms, religion, ethnicity, as well as age and gender. Test developers make efforts to 


Table 10.4 Summary of “Target dimensions and constructs # items | Alpha 
reliability coctficients Overall: Collaboration 8 887 
Overall: Problem solving 12 .907 
Problem solving: Defining the problem 6 .823 
Problem solving: Finding the solution 6 .871 
Overall: Respect scale 10 .828 
Overall: Self-awareness 12 .837 
Self-awareness: Self-management 6 .154 
l Self-awareness: Perspective taking 6 .138 


?The Rasch model requires that item parameters remain fixed irrespective of the persons being 
measured; and vice versa, person parameters remain the same irrespective of items being used 
to measure. This property of the Rasch model allows for development of common metrics 
and criterion-referenced interpretation of individual performance. Fit of data to a model is required 
to justify use of Rasch analysis to develop performance measures and to compare individuals’ 
performance across different subgroups in a population. 
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design assessments in ways that will avoid differential bias among groups. 
Notwithstanding, it is also necessary to check whether such bias may have occurred 
after the fact. The results and information from Differential Item Functioning (DIF) 
analysis provide a rich source of information for exploring the possibility of bias of 
measurements across groups. 

Since the ALIVE tool was used across three countries and four education juris- 
dictions, each of which varies in both explicit and subtle ways, it was important to 
check for DIF. Analyses were conducted across the four jurisdictions to provide 
insights into whether items functioned differently or similarly across the educa- 
tional jurisdictions. Detection of DIF was undertaken through visual inspection of 
the results from scatterplots using item thresholds derived from the Rasch model. 
For each of the three over-arching constructs (collaboration, problem solving, 
respect) and the two dimensions of self-awareness (self-management and perspec- 
tive taking), item thresholds for each of the study jurisdictions were placed on the 
Y-axis and the regional item thresholds—all four jurisdictions together—were 
placed on the X-axis. In addition, scatterplots for each of the study jurisdictions 
against each other were analysed. Figure 10.1 provides an example that shows least 
differences obtained for a jurisdiction against the region, while Fig. 10.2 provides 
two examples that illustrate greatest differences found of all comparisons, and this 
time between two jurisdictions. 

The comparison of thresholds in the Fig. 10.2 scatterplots illustrates how slight 
the differences are, even in these cases which are the most extreme. Two scatterplots 
are provided, both for the self-management dimension of self-awareness, and with 
each comparing one of the Tanzania jurisdictions with Kenya. Looking at both scat- 
terplots is instrumental in demonstrating the resilience of the patterning of the items, 
notwithstanding their elevation varying. 

As shown, there is negligible differential item functioning in the four constructs 
across the four jurisdictions. In exploring the slight differences that do occur, it is 
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Fig. 10.1 Scatterplot of problem solving item thresholds: Zanzibar against region 
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Fig. 10.2 (a) Scatterplot of self-management item thresholds: Zanzibar against Kenya. (b) 
Scatterplot of self-management item thresholds: Tanzania mainland against Kenya 


clear that they are due primarily to group difference in performance rather than bias. 
Overall, items for all constructs pattern very similarly across the four jurisdictions. 
Where differences occur, these are in terms of levels of proficiency rather than pat- 
terning differences which might imply bias. 


10.3.2.4 Item Spread 


‘Person-ability maps’ based on the Rasch model provide a view of how well items 
cover a range of proficiencies via their physical separation on a graph. Such a map, 
or graph, shows whether items are separated enough to differentiate between the 
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performance of different respondents. The person-ability map allows for the place- 
ment of items and persons on the same scale. Naturally, the higher a person’s ability, 
the greater the probability of appropriately answering an item or demonstrating a 
higher level of proficiency. 

The patterns demonstrated by the person-ability maps validate the approach to 
item design, allowing for identifiable differences in responses and informing their 
coding. The clear separation of response codes across the items for all the constructs 
demonstrates that the coding rubrics indeed captured similar degrees of discrimina- 
tion across proficiencies for most items. An ideal test would be characterised by 
items distributed right across the possible range of person abilities. Such an instance 
would allow for optimal differentiation of one person’s abilities from another. In 
these cases, there is sufficient delineation between each coding level to justify the 
attribution of descriptive scoring statements for the various categories of proficien- 
cies. The task and item design for all constructs was such that three or four levels of 
response were provided for through scoring rubrics. That the categories of responses 
in the main adhere to these levels is indicative of the robustness of the design and of 
the rubrics. The maps indicate how the response categories tend to pattern, and sup- 
port the creation of the descriptive statements that are strongly aligned with the 
original scoring rubrics. Hence, both the content of the rubrics and how the resulting 
category responses are located in the graph space provide confidence in setting cut- 
offs which are then used to identify ranges of performance within achievement lev- 
els with corresponding descriptive statements. 

The person-ability maps for the four constructs (see Figs. 10.3, 10.4, 10.5, and 
10.6) represent the coded responses of adolescents through low to higher perfor- 
mance levels. The horizontal blue lines on each map identify the approximate loca- 
tion of the cut-offs. Each map provides an illustration of the capabilities of the 
adolescents in the context of the demands of the tasks. The distribution of adoles- 
cents is on the left-hand side of the graph, and on the right are the item numbers and 
coding according to the partial credit model. Each item and its label appear at a 
position horizontally parallel to those adolescents who have a 50% chance of dem- 
onstrating that particular item’s performance. These adolescents have increasing 
probabilities of being able to demonstrate the proficiencies represented by item by 
category responses, the lower these appear on the graph. 

Along the right-hand side of each graph is the ‘logit’ scale, showing the numeric 
equivalents of the graphed locations. Each item identified along the bottom of the 
graph is represented at the different levels of quality at which adolescents respond 
to that item (shown as Cat1, Cat2, Cat3). For example, in Fig. 10.3, the lowest level 
response to item CT11, Cat1 (at a logit of about —1.8), represents the lowest level 
of quality response for the item, while Cat3 (at a logit of about 2.3) represents the 
highest quality response. The graph also illustrates how different items vary in dif- 
ficulty. For example, item CT13 at the far right of the graph is the ‘easiest’ item for 
adolescents to demonstrate overall. Information about the coding rubrics for the 
constructs can be found in this volume (Scoular & Otieno, 2024; Chap. 6, this vol- 
ume; Ngina et al., 2024; Chap. 5, this volume; and Care & Giacomazzi, 2024; Chap. 
4, this volume). 
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Fig. 10.4 Person-ability map for problem solving 


The figures provide additional information about how items contribute to sub- 
skills, and how these subskills are variably easier or more difficult for adolescents 
to demonstrate. For example, CT13 and CT63 are items that target working together 
in collaboration. Together these are the activities that more adolescents find easy to 
engage in, as distinct from communication (CT 11, 41, 61) and negotiation CT 12, 
42, 62) which cover a broader and more complex range of performance. 

Figure 10.4 reveals a less even distribution of proficiencies within each item for 
problem solving. Items contributing to the dimension of defining the problem (PS 1b, 
3b, 4b, 1c, 3c, 4c) cover a wide range of proficiencies due to the highest level of 
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Fig. 10.6 Person-ability map for self-awareness (including both self-management and 
self-awareness) 


responses presenting considerable difficulty for respondents. Ideally the task and 
item design could include additional features which would allow for more nuanced 
responses between the highest and middle categories. The remaining items which 
contribute to exploring the solution cover a relatively narrow range of proficiencies. 
Again, the task and item design needs some improvement in order to capture more 
differentiated abilities for this dimension. 

Figure 10.5 illustrates the perspectives associated with respect expressed by the 
adolescents. It should be kept in mind that the measurement of respect was limited 
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to respect for self and others, and did not include respect for property and the envi- 
ronment (see Ngina et al., 2024; Chap. 5, this volume). Similarly, the mode of 
assessment did not allow for identification of the subskills, that would separate 
respect for self and respect for others. 

Figure 10.6 shows location of items across the range of responses to the self- 
awareness tasks and items. The graphic divides items that contribute to the two 
dimensions for ease of reference. The pattern of items hypothesised to assess ado- 
lescent’s self-awareness indicates how these two dimensions—self-management 
and perspective taking—draw on adolescents’ capacities differently, notwithstand- 
ing their strength of association (r = 0.668). Although the two scales demonstrate 
high reliability, contributing robustly to the over-arching skill, perspective taking 
(SAlb, SAle, SA3a, SA3c, SA4c, and SA7d) appears slightly more difficult to 
demonstrate than self-management (SA1d, SA4b, SA6b, SA6c, SA7b, and SA7c). 
Therefore, reporting results for each scale separately provides useful information to 
inform teaching and learning in the education context. 


10.3.2.5 Summary of Approach 


The Rasch model was used to calibrate the participants’ response data and estab- 
lish the ability and item difficulties on a common scale. This approach ensured that 
item parameters were independent of the participants being measured and estimates 
of participant abilities were independent of the items used to measure the underly- 
ing ability/trait. The fit statistics were used to investigate how well the data met the 
model requirements, and misfitting items or persons were identified and addressed 
to ensure the validity of the results. Differential item functioning was used to detect 
differences in item performance for individuals with the same underlying ability but 
belonging to different subgroups, such as gender, education level, and adolescent 
age. The presence of DIF was explored to establish the scope of use and the gener- 
alisability of the results across and within countries and to distinguish between bias 
and real differences in ability distribution. The Rasch model analysis outputs pro- 
vided tools for interpreting skills that underpinned constructing and developing 
empirical proficiency levels, which described increasing competency levels for the 
over-arching constructs. 


10.3.3 Proficiencies Distributions across Jurisdictions 


ALiVE reports adolescents’ demonstration of their life skills and values through brief 
statements. These statements make explicit how the adolescents actually respond—as 
opposed to reporting scores, for example. Varying across the four constructs, there are 
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three or four statements which describe increasing levels of development or profi- 
ciency. These descriptions are based on analysis of the items that fall within the vari- 
ous cut-offs shown in Figs. 10.3, 10.4, 10.5, and 10.6. All items were designed such 
that adolescents could respond to these at varying level of proficiency or quality which 
were captured through the scoring rubrics. Reviewing the behaviours targeted by 
those rubrics against placement of the items in the person-item maps acts as a confir- 
mation of the increasing levels of quality. Adolescents performing within each of the 
descriptive levels are shown in Tables 10.5, 10.6, 10.7, 10.8, 10.9, and 10.10. 

In this section the distributions of adolescent proficiencies for each construct are 
shown in the context of jurisdictional differences. As pointed out in the presentation 
of the DIF results, there are no differences across Kenya, Tanzania (mainland and 
Zanzibar), and Uganda in terms of how items contribute to the scales—this is uni- 
formly consistent. Some differences in distributions of the proficiencies are 
highlighted. 


10.3.3.1 Collaboration 


Four levels of performance describe what adolescents were able to demonstrate dur- 
ing the collaboration tasks (Table 10.5). Overall, most adolescents were attentive to 
the discussions; they queried the views of others and engaged actively in the perfor- 
mance tasks. Relatively few either did not engage visibly (9.5%) or prompted others 
to engage (10.0%). 


Table 10.5 Collaboration: descriptive statements and adolescents’ proficiencies 


Kenya | Tanzania Mainland | Uganda | Zanzibar | Regional 


Collaboration proficiency levels % of N = 20,116 adolescents 


Level 1: does not engage either by Ta 13.1 9.4 10.3 9.5 
being attentive to discussion, 
speaking, or through action 
Level 2: is attentive to the 44.7 42.2 50.2 44.7 44.8 
discussion and may query the 
views of others, but does not 
contribute in word or action 
Level 3: collaborates through 37.7 33:3 34.2 32.9 35.7 
speaking and being attentive in 
discussions, and engaging actively 
in performance tasks 

Level 4: collaborates through 10.4 11.4 6.1 12.1 10.0 
taking positions and contributing 
ideas, prompting others, and being 
attentive to others’ inputs 
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Table 10.6 Problem solving: descriptive statements and distributions of proficiencies 


Problem solving proficiency levels 


Level 1: struggle to recognise a 
problem or its nature and therefore 
unable to identify possible solutions 


Kenya 


Tanzania Mainland 


Uganda 


Zanzibar 


Regional 


29.1 


% of N = 45,442 adolescents 


38.3 


33.1 


22.6 


32.9 


Level 2: able to recognise existence 
of a problem from one perspective, 
and act on that to identify a possible 
solution 


SkA 


43.0 


52.7 


43.6 


49.1 


Level 3: able to recognise existence 
of a problem from one perspective, 
able to identify a main approach 

to solving the problem, and can 
justify it 


15.0 


11.5 


19.7 


12.9 


Level 4: able to recognise existence 
of a problem from multiple 
perspectives, understanding that 
there may be multiple solutions to 
evaluate and select from 


4.8 


7.6 


2 


14.0 


5.1 


Table 10.7 Respect: descriptive statements and distributions of proficiencies 


Respect levels of expression 


Level 1: unable to respond in a 
relevant way 


Kenya 


Tanzania Mainland 


Uganda 


Zanzibar 


Regional 


6.7 


% of N = 45,442 adolescents 


10.1 


5.4 


4.0 


7.4 


Level 2: aware of infringement of 
rights, or of bad behaviour by one 
person toward another but does not 
‘call it out’ 


35.6 


30.6 


371 


22.6 


34.4 


Level 3: able to interpret bad 
behaviour as lack of respect for 
others or self, and may take 
conciliatory steps to resolve 
situations 


51.6 


46.9 


51.7 


50.2 


Level 4: aware of links between 
respect for property and respect for 
person, and will act in a respectful 
way toward others and in defence of 
others and self 


6.1 


12.5 


5.8 


22.3 


8.0 
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Table 10.8 Self-management: descriptive statements and distributions of proficiencies 


Self-management proficiency Kenya | Tanzania Mainland | Uganda | Zanzibar | Regional 
levels % of N = 45,442 adolescents 
Level 1: unable to regulate 25.7 27.4 26.1 19.0 26.3 


negative emotions or responses 
Level 2: able to control self in a 55-7 40.9 53.8 45.0 50.7 
negative or stressful situation 
through repression of emotion or 
avoidance 

Level 3: sufficiently self-aware 18.7 31.8 20.1 36.1 23.1 
and confident to respond 
adaptively even when directly 
confronted or attacked 


Table 10.9 Perspective taking: descriptive statements and distributions of proficiencies 


Perspective taking proficiency Kenya | Tanzania Mainland | Uganda | Zanzibar | Regional 
levels % of N = 45,442 adolescents 
Level 1: aware of others’ 29.2 34.0 27.2 21.6 30.0 


perspectives only in relation 


to oneself 

Level 2: aware that others may be | 67.6 57.3 68.6 62.8 64.7 
impacted by multiple factors 

Level 3: aware that others act on 3:3 8.7 4.3 15.7 5.3 


the basis of multiple factors, both 
personal and community 


10.3.3.2 Problem Solving 


Four levels of performance describe what adolescents were able to do during the 
problem-solving assessments (Table 10.6). A reasonably large proportion of ado- 
lescents (32.9%) struggled to identify possible solutions to a problem, while 
nearly half (49.1%) of the adolescents were able to recognise existence of a prob- 
lem from one perspective and act on that to identify a possible solution. Relatively 
few were able to justify solutions or identify multiple approaches to solving a 
problem. There is a slight skew in the distribution of the Zanzibar adolescents, 
with fewer than expected at the lowest level, and more than expected at the 
higher levels. 


10.3.3.3 Respect 
Four levels describe how adolescents expressed respect in terms of regard for others 


and self (Table 10.7). Overall, a large proportion of the adolescents were aware of 
poor behaviour (34.4%), and able to interpret this as lack of respect for others or 
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self, with need for conciliatory steps (50.2%). However, very few were able to act 
respectfully in defence of others and self. It is noteworthy that a larger proportion of 
Zanzibar adolescents were part of these few. 


10.3.3.4 Self-Awareness: Self-Management 


Three levels of performance describe how adolescents demonstrated self- 
management (Table 10.8). Overall, the majority of adolescents (50.7%) were able to 
demonstrate how to control self in a negative or stressful situation through repres- 
sion of emotion or avoidance. They were less able to respond adaptively when pre- 
sented with situations in which they might be directly confronted or attacked. 


10.3.3.5 Self-Awareness: Perspective Taking 


Three levels of performance describe how adolescents demonstrated perspective 
taking (Table 10.9). Most (64.7%) of the adolescents were aware that others may be 
impacted by multiple factors. They were, however, less able to see views on self 
from the perspective of others. Again, there is a slight skew in results from Zanzibar 
with less than expected performing at the lowest level, and more at the highest level. 
The lower proportion of adolescents performing at the highest level is a clear indica- 
tion of the greater complexity of this skill, that perhaps requires more experience or 
maturation. 

Review of the proficiency levels across each of the constructs reveals some dif- 
ferences in distributions across the jurisdictions. In the main, these differences are 
re-aligned with reference to proximate categories. This finding supports the patterns 
revealed by the person-item maps, that adolescents across the four jurisdictions 
responded in very similar ways, differing only slightly in the actual performance 
levels (as demonstrated in particular by the smallest group, Zanzibar). These simi- 
larities provide confidence in the sequence of incremental learning steps in the life 
skills and value processes. 


10.3.4 Proficiencies by Age and Education 


Gender had no impact at any levels for any of the constructs—in other words, males 
and females performed similarly to each other across the board, regardless of jurisdic- 
tion. The disability index information presented in Table 10.3 was derived from parent 
data where these parents identified that their adolescents had ‘at least some difficulty’. 
Based on the data collected on disability, there are no associations with performance 
on the life skills and value tasks. Accordingly, detailed information on performance by 
disability is not reported here. Information on age and education status is provided 
since these two factors appear to be associated with performance. In brief, as adoles- 
cents age, and as adolescents move through education grades, their performance 
improves (Table 10.10). 
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Table 10.10 Proficiency levels of adolescents by age and education 


Level 1 Level 2 Level 3 Level 4 

Adolescents’ characteristics % of adolescents 

Collaboration 

Age group 13-14 years 11.7 47.5 34.5 6.4 
15-17 years 7.4 42.4 36.8 13.3 

Education level Primary 10.8 46.7 33.5 9.0 
Secondary 6.0 40.5 41.4 12.1 

Problem solving 

Age group 13-14 years 38.7 46.5 11.2 So 
15-17 years 27.3 51.5 14.7 6.5 

Education level Primary 38.6 47.3 10.7 3.4 
Secondary 17.5 54.4 18.6 9.4 

Respect 

Age group 13-14 years 9.2 38.5 46.1 6.2 
15-17 years 5.6 30.5 54.1 9.8 

Education level Primary 8.6 39.1 46.5 5.8 
Secondary 2.8 23.1 60.5 13.7 

Self-management 

Age group 13-14 years 29.9 50.1 20.0 NA 
15-17 years 22.7 51.2 26.1 NA 

Education level Primary 29.8 51.5 18.7 NA 
Secondary 15.8 49.7 34.5 NA 

Perspective taking 

Age group 13-14 years 35.2 60.9 3.9 NA 
15-17 years 25.1 68.4 6.6 NA 

Education level Primary 34.8 61.7 3:5: NA 
Secondary 16.8 73.5 9.8 NA 


Note: NA not applicable 


Age has an influence on the demonstrated proficiencies of adolescents across all 
the constructs. Older adolescents demonstrate higher proficiencies compared with 
the younger adolescents. For example in the case of collaboration, 13.3% of the 
adolescents aged 15 to 17 years compared with 6.4% of the adolescents aged 13 to 
14 years, collaborated through taking positions and contributing ideas, prompting 
others, and being attentive to others’ inputs (Level 4). On Level 1, 7.4% of adoles- 
cents aged 15 to 17 years compared with 11.7% of the adolescents aged 13 to 
14 years did not engage either by being attentive to discussion, speaking, or 
through action. 

Education level also is associated with increasing proficiencies. More educated 
adolescents demonstrated higher proficiencies compared with the less educated 
adolescents. For example in self-management, 34.5% of the adolescents who have 
reached secondary level of education compared with 18.7% of those who have 
attained a primary level of education are sufficiently self-aware and confident to 
respond adaptively even when directly confronted or attacked (Level 4). On Level 
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Fig. 10.7 Associations between age, education, and proficiencies 


1, 15.8% of the adolescents with a secondary level of education compared with 
29.8% of the adolescents with a primary level of education are unable to regulate 
negative emotions or responses. 

Although there are obvious associations between age and education level, it is 
the latter that appears to be more strongly associated with increasing proficiencies, 
as illustrated in Fig. 10.7. There is no doubt that adolescents currently studying in 
secondary school demonstrate higher proficiencies than do those still in the primary 
years. Whether this is due to the effect of schooling or to other factors, uncontrolled 
for, that characterise the two groups, is not known. The primary school attendees 
account for more than double the sample size of the secondary (see Table 10.3) but 
both groups are very sizable, so the difference cannot be attributed to the variability 
that might be associated with small sample sizes. There is also a general flattening 
effect over age for the primary school attendees, and for the self-management 
dimension of self-awareness across the full sample. 


10.4 Discussion 


ALIVE developed an assessment of three life skills and one value, creating a tool 
that gathered responses from adolescents to a variety of scenario-based and perfor- 
mance tasks. The open-ended responses of the adolescents were coded according to 
rubrics that allowed for evaluation of levels of quality in those responses. The coded 
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data were then analysed according to their hypothesised contributions to over- 
arching constructs, and in some cases to dimensions and subskills. The aim was to 
develop a measure that would generate information about what adolescents are able 
to do in terms of collaboration and problem solving, and how they perceive 
themselves and others around them in terms of respect and self-awareness. 
Scale reliabilities and person and item fit statistics calculated from the collected 
data support the validity of the assessment for its intended purpose. Given the com- 
prehensive and systematic sampling, generalisability of the results can reasonably 
be claimed. To date, there has been no other study to collect evidence of life skills 
and values at large scale through household-based assessment. The initiative dem- 
onstrates that robust and useful tools can be developed for use outside of the formal 
classroom space to generate data that is useful within that space. 

There are limitations to note. These include impact of the chosen medium of 
assessment (oral one-to-one administration in the household), use of coding rubrics, 
training of Test Administrators, use of local language translations. The impact of 
some of these limitations is unknown and will require more controlled assessment 
environments in order to ascertain. However, the matters of range of targeted profi- 
ciencies, and training of the Test Administrators, are briefly discussed in this section. 

The tools are simple in structure and the approach to coding of adolescent 
responses is not highly nuanced. Scoring rubrics were sufficiently behaviour- 
focussed to enable Test Administrators a high degree of reliability in coding. 
Decisions about the nature of the tasks themselves and the decision to code across 
only three or four levels were made in the light of the logistics and realities of 
household-based assessment in low resourced environments. These decisions 
have had the consequence that more finely delineated differences in proficiencies 
between adolescents are not accommodated in the current descriptive statements. 
Development of more assessment tasks and more detailed rubrics could be mod- 
elled on what has been used to date in order to increase the range of information that 
can be captured. In ALiVE all adolescent respondents engaged with at least three of 
the constructs. This meant reduced assessment time available for any one of these 
and so led to pragmatic decisions about complexity of tasks and coding. 

The training of Test Administrators was undertaken with a similar approach 
across jurisdictions. However, there were some differences in criteria used to recruit 
and select these, and the actual training events were particular to each jurisdiction. 
Due to this potential variability, and also to general concerns where ratings or per- 
formance are required, it is possible that assessor bias might have influenced the 
assessment process either on a case to case basis, or through slightly different juris- 
dictional approaches to the training. The interaction between Test Administrators 
and adolescents also could influence the confidence of the adolescent, who typically 
was engaging in an unfamiliar process—notwithstanding that the content of the 
assessment tasks themselves was familiar, being based in daily life. The extent to 
which this possibility could have led to under-estimates of real proficiencies is not 
known. Further research with the ALIVE tools could focus on investigating issues 
of bias and interaction modes. 
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Another potential influence on adolescents’ capacity to respond at their most 
proficient level was language. The tools were translated into several local languages 
commonly spoken in the sampled districts, and adolescents were given the choice of 
language in which to interact. During the assessment, Test Administrators would 
use a language with which the adolescent was most comfortable, typically either 
English or the local language. After electing for one language, in some cases 
adolescents requested to switch to another to comprehend the tasks better. 
Another aspect of the language issue pertains to the actual translations of the tool. 
These were not subjected to back translation as a standard process, and different 
quality assurance approaches were used. Further research under more controlled 
conditions could explore the influence of language on assessment results. 

The creation, development, and use of ALiVE’s tool was accomplished through 
the efforts of teams across the three participating countries (Turner et al., 2024; 
Chap. 11, this volume). One outcome of these efforts was the assessment of over 
45,000 adolescents, generating data on life skills and a value to inform curricular 
and assessment needs of the four educational jurisdictions involved. Another out- 
come was the confirmation that life skills and values can be captured through a 
household-based assessment model, providing a template for the future. 


Appendix 1: Item Fit Statistics 
for the Over-Arching Constructs 


(a) Item fit statistics for collaboration 


Unweighted fit Weighted fit 
Item Fit value t-stat p-value Fit value t-stat p-value 
CT11 0.98 —2.17 0.030 0.99 —1.16 0.247 
CT12 0.96 —4.19 0.000 0.97 —2.65 0.008 
CT13 0.92 1.72 0.000 0.94 —6.09 0.000 
CT41 0.99 —1.37 0.170 1.00 0.10 0.923 
CT42 0.98 —1.46 0.146 1.01 0.61 0.544 
CT61 0.94 —5.20 0.000 0.97 —3.07 0.002 
CT62 1.14 9.67 0.000 1.14 13.71 0.000 
CT63 1.04 2.81 0.005 1.06 5.57 0.000 


(b) Item fit statistics for problem solving 


Unweighted fit Weighted fit 
t-stat 


Fit value 


Item Fit value 


0.000 


PSIc 1.04 6.08 0.000 1.06 8.36 0.000 
PSid 1.04 Bee) 0.000 1.05 7.03 0.000 
PSle 1.11 11.41 0.000 1.10 15.99 0.000 


(continued) 
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Unweighted fit Weighted fit 
Item Fit value t-stat p-value Fit value t-stat p-value 
PS3b 0.98 —2.53 0.011 1.01 0.95 0.341 
PS3c 0.98 —2.43 0.015 1.01 0.72 0.472 
PS3d 0.92 —9.67 0.000 0.95 —8.23 0.000 
PS3e 0.87 —15.48 0.000 0.91 —15.47 0.000 
PS4b 0.87 —14.62 0.000 0.91 —14.30 0.000 
PS4c 1.00 0.28 0.777 1.05 7.86 0.000 
PS4d 0.88 —6.31 0.000 0.98 —2.78 0.005 
PS4e 0.91 —5.44 0.000 1.00 0.56 0.579 
(c) Item fit statistics for respect 
Unweighted fit Weighted fit 
Item Fit value t-stat p-value Fit value t-stat p-value 
RTla 0.94 —10.11 0.000 0.95 —8.74 0.000 
RT1b 0.94 —9.50 0.000 0.95 =113 0.000 
RTid 0.97 =335 0.001 1.00 —0.92 0.359 
RT2a 1.11 14.42 0.000 1.09 16.15 0.000 
RT2b 1.07 8.46 0.000 1.06 10.14 0.000 
RT3a 0.95 =1,53 0.000 0.97 —5.91 0.000 
RT3e 0.95 —8.37 0.000 0.96 —6.59 0.000 
RT6a 1.04 5.86 0.000 1.04 6.42 0.000 
RT6b 0.95 —7.47 0.000 0.97 —5.66 0.000 
RT6c 1.08 11.15 0.000 1.08 13.19 0.000 
(d) Item fit statistics for self-awareness 
Unweighted fit Weighted fit 
Item Fit value t-stat p-value Fit value t-stat p-value 
SA1b 0.99 —2.13 0.033 0.99 —1.85 0.065 
SAle 1.06 7.73 0.000 1.07 11.10 0.000 
SA3a 1.05 7.89 0.000 1.04 7.08 0.000 
SA3c 0.95 —7.40 0.000 0.97 —3.88 0.000 
SA4c 0.96 —6.39 0.000 0.97 —4.31 0.000 
SA7d 0.94 —8.61 0.000 0.96 —6.03 0.000 
SAld 1.01 1.79 0.074 1.01 1.76 0.079 
SA4b 1.05 8.31 0.000 1.05 7.81 0.000 
SA6b 1.02 2.57 0.010 1.02 3.10 0.002 
SA6c 1.01 0.69 0.493 1.01 1.75 0.081 
SA7b 1.00 0.59 0.553 1.00 —0.05 0.962 
SA7c 1.01 1.65 0.099 1.02 2.65 0.008 
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Chapter 11 A 
Reflections on ALiVE’s Collaborative gree 
Endeavour 


Fergal Turner, Michael Babu, and Olivia Mcintire 


Abstract AliVE’s visible achievement has been to develop contextualised assess- 
ment tools for three life skills and one value, undertake a large-scale assessment 
program at household level across three countries, and engage with hundreds of 
stakeholders in the process. What is less visible is how this has been achieved, who 
is responsible for it, and what the motivating force behind it has been. This chapter 
describes reflections on the collaborative processes that underlie these activities. In 
so doing, the chapter locates that collaboration within the history of community and 
civil society contributions to education in East Africa, and more specifically within 
Kenya, Tanzania, and Uganda. The chapter’s reflections make explicit who the con- 
tributors were and how they were able to work together. In these reflections two 
factors are of particular interest. The first concerns the link between the way indi- 
viduals and organisations worked together, and the actual life skills that were the 
object of their attention, in particular the skill of collaboration. The second is the 
constituting of the endeavour as a learning journey. The process is seen not merely 
as a production of an assessment tool and consequent results which can be used to 
advocate for life skills in education, but as a vehicle for equipping collaborators 
with the technical and work skills that they can take forward into future educa- 
tion spaces. 


11.1 Introduction 


Collaboration has many places in the heart of the ALIVE program. First, it sits at the 
heart of ALiVE’s understanding of lifeskills and values, as one of the constructs 
contextualised and measured by the program. Second, it sits at the heart of ALiVE’s 
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orginal theory of change, as a prioritisation of collaborative action to develop con- 
textualised assessments, evidence led advocacy, and a learning community. Finally, 
collaboration is the foundation which ALIVE stands on. ALIVE was conceptualised 
as a program of the East African Regional Education Learning Initiative (RELD, 
which is a learning community of more than 70 organisations working on education 
in Kenya, Tanzania, and Uganda. ALIVE has been put into action by the Values and 
Lifeskills cluster of RELI, meaning that it is not a program of one organisation, but 
of many working together. 

This kind of collaborative action in assessment is new in many ways. It gives 
equal priority to contextual, experiential, and technical knowledge, rather than plac- 
ing priority on the latter. It also prioritises negotiation and consensus. The hypoth- 
esis is that by working collaboratively, ALIVE can go further in achieving its three 
goals than it could ever have done if it had been delivered by one organisation. The 
successes of ALIVE in achieving its three goals is undisputed and is given space in 
the rest of this volume. In this chapter we explore what collaboration meant for 
ALiVE, and what the endeavour can tell us about the future of collective action in 
education. 


11.1.1 The Nature of Inter-Organisational Collaboration: 
Benefits and Challenges 


Dillenbourg (1999) holds that collaboration is more likely to occur in situations 
where the agents or partners have relatively similar levels of expertise and power 
relations. Collaboration should not be dominated by a single party’s views since this 
can compromise mutual engagement, interdependency, and interactivity (Chrislip & 
Larson, 1994), and can also result in command and control behaviour instead of 
creating a shared vision and goals (Anslinger & Jenk, 2004). Mutual engagement 
and interdependency can only thrive if participants are willing to share their ideas, 
listen, and learn from others (Lai, 2011; Chrislip & Larson). 

Inter-organizational collaboration offers various benefits, some of which are 
more explicit than others. Collaboration can provide a structure for transmitting 
knowledge and managing responsibilities, as noted by Kosmutzky and Putty (2016). 
In addition to these tangible benefits, collaboration can offer social benefits such as 
building professional networks, gaining friendships, and a sense of self-fulfillment 
for contributing to a common goal (Ulnicane, 2015). 

The nature of collaboration itself can have a significant impact on a project’s 
success. Ulnicane (2015) emphasized that regular communication among partners 
is crucial since the project outcomes depend on the contributions of partners with 
diverse expertise. Selden et al. (2006) observed that the greater and more intense the 
involvement of partners in collaboration, the better the implementation, and greater 
the ownership of the project objectives. Leahey (2016) notes that successful 
collaboration requires an organized series of activities that focus on interpersonal 
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relationships, as well as the operating systems of the partnering organizations. 
The crucial factor is the identification of a common purpose. 

Collaborating across teams of experts can bring about various benefits, but it also 
comes with some drawbacks. One disadvantage is that individual contributions may 
go unnoticed as recognition is often given to the whole team (Kosmutzky & Putty, 
2016). Moreover, collaborative processes can be time-consuming and require regu- 
lar coordination and communication, which may be challenging for a diverse team 
that lacks intercultural and interpersonal competencies (Dusdal & Powell, 2021). 
Another challenge is the need for planning and technological support to promote 
trust and sensitivity among team members who might be working from different 
locations (Livingston, 2003). Cummings and Kiesler (2005) add that spatially 
spread networks require significant coordination efforts to bring together ideas and 
expertise. 

Chrislip (2002) highlighted the importance of creating a collaborative environ- 
ment that allows participants to be part of the process and gives them a sense of 
buy-in. This environment is characterized by three components: step-by-step dis- 
cussion of goals and benefits, clarification of rules and guidelines around open com- 
munication to obtain feedback from each party, and anticipation of possible 
challenges. Finally, the different operating contexts of each partner can pose a chal- 
lenge to collaboration, as socio-cultural, political, and economic differences may 
affect the partners’ ability to provide resources and fully participate in project activ- 
ities (Anslinger & Jenk, 2004). This challenge can be addressed by clearly defining 
and agreeing upon the requirements and expectations of each partner during the 
planning stage. 


11.1.2 Historical Context for Civil Society Collaboration 
in East Africa 


On a global scale, civil society actors have long been involved in the field of educa- 
tion. Between 1950 and 1990, there was an unprecedented expansion of interna- 
tional non-government and civil society action in the field (Mundy & Murphy, 
2001). The World Conference on Education for All (WCEFA) held in Jomtien, 
marked a new age for non-government actors in international educational coopera- 
tion and a recognised role in the provision of global educational services. While the 
WCEFA is considered by many to have encompassed a Western-led agenda which 
emphasized a North to South delivery of education, the conference triggered an 
international network of civil society actors to come together to reshape the educa- 
tion landscape. 

In 1999, the non-government organisations Oxfam, ActionAid and Education 
International formed the Global Campaign for Education (GCE) in preparation for 
the World Education Forum in Dakar which in part brought civil society organisa- 
tions to the table to influence the global education agenda (GCE, 2022). Today, the 
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GCE comprises over 100 national and regional education networks and interna- 
tional organisations, representing thousands of Civil Society Organisations (CSOs) 
as well as women’s groups, parents’ associations, teachers’ organisations, child 
rights campaigners, youth groups and academic institutions. 

Since Kenya’s independence in 1963, community-based organisations (CBOs) 
have been key players in the development of education provision in Kenya (Mundy 
et al., 2010). The period preceding the 1992 multiparty declaration was a significant 
time for civil society to unite as it rallied against the one-party state in power. Civil 
society such as the Women’s Movement, Christian organisations, university stu- 
dents, and the rural poor, ran opposition campaigns, which when backed by the 
international community, brought about a historic multi-party election (Muthoni 
Githuku, 1993). In 1999, the ElimuYetu Coalition was formed, which brought 
together education-focused civil society and helped influence the abolition of ele- 
mentary school fees (Mundy et al., 2010). The coalition is still active today and acts 
as the national platform for civil society organisations (CSOs) and non-state educa- 
tion stakeholders to lobby for education for all in Kenya (Elimu Yetu Coalition, 
2022). Similarly, the 2002 elections also marked a major turning point where CSOs 
were permitted to develop and hold more recognized space in the Kenyan educa- 
tional landscape. By 2005, the Kenya Education Sector Program aimed to bring 
local organisations and international organisations together under a government 
policy (Riddell, 2007). 

After Tanzania’s independence, the country entered a period of forming a new 
nation, guided by President Julius Nyerere’s philosophy of African socialism, which 
focused on the creation of a self-sufficient welfare driven society. From 1960 to 
1980, the government invested a lot into education initiatives to address the fact that 
only about 20% of the population had received or were in formal education. From 
1990 to 2000, there was an unprecedented expansion of NGOs and CSOs address- 
ing social issues, including educational provision. In particular, the multi-party 
elections in 1995 opened up space for civil society at the national level (Simon & 
Sikoyo, 2021). In 1999, the Tanzania Education Network/Mtandao wa Elimu 
Tanzania (TEN/MET) was founded to coordinate civil society stakeholders to influ- 
ence basic education policy and practices. Notably, they joined the Global Campaign 
for Education in 1999. However, in 2010 there was a shift in the government’s atti- 
tude towards civil society, which inhibited civic space and withdrew the gains that 
had been made over the previous 10 years. It has only been in the past 3 years that 
civil society and government have been in partnership to address social issues. 
During this period, TEN/MET continued to bring together CSOs in the educational 
sector and today are still the key network in Tanzania, comprising 158 education 
organisations which work together to improve education policy and practice (TEN/ 
MET Coordinator, personal communication, 2022). 

Civil society in Uganda played a critical role in the country gaining indepen- 
dence in 1962. However, not long after, civil society organisations were brought 
close to the state to ensure they were operating in the interest of the government 
(de Coninck & Larok, 2021). From 1986, Uganda saw a growth of civil society 
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organisations, backed by international donor funding. The government did not inter- 
fere with their work as long as they had non-political agendas and focused on the 
provision of social services including education. However, in the late 1990s, the 
state redefined the space that civil society was permitted to occupy, ensuring they 
operated under supervision of the Ministry of Internal Affairs (de Coninck). Within 
this new restrictive civic space, education-focused NGOs still managed to form the 
Forum for Education NGOs in Uganda (FENU) in 2001 to bring together CSOs and 
CBOs, and other education stakeholders to advocate for quality education 
(FENU, 2022). 

RELI was formed in 2017 as an innovative network that unites more than 70 East 
African organisations to strive for improved learning outcomes in Kenya, Tanzania, 
and Uganda. RELI has a wide array of achievements such as increasing inclusion of 
learners from conflict-affected areas in Uganda, engaging in curriculum focused 
policy advocacy in Kenya, and building the capacity of Teacher Development pro- 
fessionals in Tanzania. 

This historical context shows the long journey towards more collaboration in 
education by civil society actors in East Africa. It is this historical trajectory which 
created the fertile ground for ALIVE as a collaborative endeavour. 


11.1.3 Defining Collaboration in ALIVE 


The title of this chapter refers to ALiVE’s collaborative endeavour. How can we 
define collaboration in the context of a program such as ALIVE? ALIVE is a proj- 
ect led and delivered by several organisations working together. These organisa- 
tions range in size and mandate and are represented in the ALiVE program by 
individuals with diverse backgrounds and technical specialisations. 

A study (Nansubuga et al., 2023) which looked at the inter-organisational col- 
laboration in ALIVE asked the questions: what motivated collaboration in ALIVE? 
What challenges did the collaboration process face, and how were they mitigated? 
And, what matters most in ensuring and sustaining success in collaboration? 

The findings show that motivation to collaborate requires clear vision, team- 
work, learning, leadership, collective decision-making, and commitment. 
Challenges faced in collaboration were interdependence (and the pressure it 
entails), unlearning to re-learn (that is the process of moving away from bias and 
preconceived ways of doing), and joint ownership (the lack of independence). 
Finally, building on both the motivators and the challenges, the study showed that 
the three most important activities were defining goals, defining how (ways of 
working together), and maintaining consistency and continuity in participation. 
This framework is shown in Fig. 11.1. 

Building on this framework and drawing from the technical discussion on col- 
laboration as a skill, presented in Scoular and Otieno (2024; Chap. 6, this volume), 
an operational definition is used in this chapter: 
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Step 1 Step 2 Step 3 
defining the goals defining how continuous participation 


TERA and values 


e Identify the 
curriculum needs 
of each partner 

© Agree on the 
goals and 
objectives 

e Clearly define 
objectives at 
every stage of the 
collaboration 
process 

Define 
personalised 
developmental 
goals and 
benefits 


Induct the 
participants as they 
come on board 
Agree on how to 
work 
collaboratively 
Contextualise the 
assessment of the 
competences as 
they are 
understood in East 
Africa 

Train participants 
in the various 
analysis methods 
used to develop the 
tools 


Bring together people 
with diverse abilities to 
achieve a desired goal 
Value each one’s ability 
as equally important 
Keep engaging 
participants to get 
feedback on the quality 
of the tool 

Keep modifying the tool 
and help others to adopt 
the changes 
Acknowledge each 
individual’s strength 
Promote mutual 
interaction to get work 
done 

Remind each other of 
the purpose of the 
collaboration 


e Remain accountable to 
the project objectives 

© Be passionate about the 
ultimate goal to reduce 
fear of setback 

© Be open minded to let in 
others’ views and to learn 
from them 

e Remain flexible and 
ready to re-learn 

Agree to disagree but 
with respect 

© Show respect to others 
regardless of who you 
are. 

e Maintain trust to increase 
confidence 

© Be empathetic to others 
to accomplish tasks 

© Be cooperative to support 
learning 


Unlearning to re-learn 


A SUCCESSFUL AND SUSTAINABLE COLLABORATION PROCESS 


Fig. 11.1 Framework for collaboration in practice (Nansubuga et al., 2023) 


Collaboration is the process of two or more people working together on a common task to 
realize shared goals, and involves a high level of negotiation, interactivity, interdependence, 
taking leadership, accommodating each other, consensus building, and division of labor in 
a group setting. (ALIVE, 2020) 


11.1.44 Questions for the Chapter 


Having reviewed the history and role of civil society in education in East Africa, we 
now turn to ALIVE. Using qualitative data gathered from a process of reflection 
engaged in by ALIVE members, the following questions are addressed: 


(a) How is ALIVE aligned with collaboration and its subskills as they were defined 
by the contextualisation process? 

(b) What were the perceived benefits of collaboration in achieving ALiVE’s goals? 

(c) What was learned about how to put collaboration into practice? This includes: 


(i) What worked for building collaboration in ALiVE? 
Gi) What challenges were faced? How were they overcome? 
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11.2 Methods 


11.2.1 Approach 


Data were collected as outputs of a reflective exercise undertaken by the ALIVE 
team between July and November 2022. The purpose of the process was to articu- 
late and document what had been learned during the first 3 years of the ALIVE 
program. 

The learning process was directed by a steering committee consisting of Regional 
Learning Leads (Uwezo Uganda and Zizi Afrique), National Learning Leads 
(Strathmore University in Kenya, GLAMI in Tanzania, and Luigi Giussani Institute 
for Higher Education in Uganda), and an external Global Learning Partner (Oxford 
MeasurEd). The process was facilitated by the Global Learning Partner, but all key 
decisions relating to research questions, research design, analysis and synthesis 
were taken by the Regional and National Learning Lead organisations. 

At the beginning of the process, the global learning partner developed a toolkit 
to structure individuals’ reflections on the ALiVE program. This toolkit focused on 
defining and describing ‘moments of learning’. These moments were defined as a 
specific incident or event which triggered a realisation or new insight. The toolkit 
was based on a short survey which was distributed by national learning leads to col- 
lect data from across the three countries. The result of this was over 100 insights 
reflecting what has been learned over the course of ALiVE’s development and tech- 
nical activities. 

Analysis workshops were convened to analyse the data from the surveys. These 
workshops were attended by individuals who had been involved in the ALIVE pro- 
cess to varying degrees, from technical team members to government counterparts 
and other organisations working on similar programs. All who took part in the anal- 
ysis workshops had also submitted their reflections, though not all who submitted 
reflections took part in the analysis. 

Once detailed notes from the three analysis workshops had been collected, the 
national learning leads synthesised findings across the three countries. 


11.2.2 Participants 


The participants from whom data were collected were all individuals who had 
played an active role in the delivery of the ALiVE program. This included the tech- 
nical team members, as well as national advisory committee members and others 
who had been involved during the process. A summary of these participants and 
their affiliations is shown in Table 11.1. 
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Table 11.1 Participants in ALIVE reflective learning process 


Kenya Tanzania Uganda 
Engagement of contributors 
Contributed data 39 28 27 
Invited to participate in analysis 39 15 21 
Affiliations of contributors 
Civil society Organisations 24 20 19 
Government institutions 8 4 
Academic institutions 3 2 7 
Teachers 4 2 0 


11.2.3 Study Limitations 


It should be noted that this study did not constitute a formal evaluation of the effec- 
tiveness of the ALIVE program. Rather it was an exercise in collective reflection, 
which placed the experience of individuals at the centre of the data collection and 
analysis process. This approach was chosen to build a collective, shared understand- 
ing of the origins and path of the program, and where it might go in the future, rather 
than to build a robust evidence base on what had worked for ALIVE. 

Questions of time and recall were important limitations to the data collected. 
Many participants struggled to recall precise details of events which had taken place 
in the past. This means that some of the broad findings presented here are missing 
fine grained explanations or examples. 


11.3 Findings 


Figure 11.2 presents the range of organisations and their roles in the ALiVE pro- 
gram. The figure is structured to show the comparative depth of collaboration of 
each of these groups, and they are categorised by those whose work is located in 
East Africa, and those whose support came from outside of East Africa. 


11.3.1 Collaboration in East Africa 


In each of the three countries, there were two important structures, technical teams 
and national advisory committees. The technical teams comprised individuals 
from the RELI member organisations, as well as individuals from other organisa- 
tions working in each country. They worked nationally and led on the 
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Collaborators in Kenya, Tanzania o Disseminating ALiVE 
and Uganda Partnership in messages 
delivering assessments 


Support with building 
community support 


Advice and collaboration 
throughout process 
National We 
We 


uonesoqgej|o) 
Jo janay 


Engagement in 


Contextualisation and tool 
development, leading data 
collection, analysis and reporting 


ALiVE Technical Teams 


2 
Close collaboration on tool Support on S A 
learning = 
Guidance and other Knowledge sharing through reports 
Collaborators globally support (passive) and conferences (active) 


Fig. 11.2 Mapping ALiVE collaboration 


contextualisation, tool development, and assessment process. Within these techni- 
cal teams there were lead organisations at both the regional (Zizi Afrique) and 
national (Zizi Afrique, Milele Zanzibar Foundation, Luigi Giussani Institute of 
Higher Education) levels. The National Advisory Committees constituted a wider 
network of individuals involved in assessment in each country. This included rep- 
resentatives from government agencies and other organisations. National Advisory 
Committees had a non-executive role in delivering ALiVE. They took part in all 
activities, but their role was to provide advice, rather than to take decisions. The 
aim was that these committees would ensure that ALIVE was both benefitting from 
a wide range of expertise and experience in assessment, but also that the program 
was connected to wider trends in the education sectors in each country. 

The organisations involved in the technical teams and national advisory com- 
mittees (Table 11.2) demonstrates the breadth of collaboration involved in deliv- 
ering ALIVE. 

The regional nature of the ALIVE program is key to its vision. The structure of 
this initiative has largely followed a regional to national to regional pathway. A 
general strategic direction is first determined at the regional level. This is then 
adapted and implemented at the national level, before being brought back and 
reflected upon at the regional level. 
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Table 11.2 Overview of technical team and national advisory committee membership 


Tanzania Uganda Kenya 

Technical teams 

Ministry of Education and Teacher & Instructor Education | RELI Secretariat 

Vocational Training and Training (TIET) Department 

(MoEVT) — Zanzibar 

Zanzibar Institute of Education Makerere University Amplify Girls 

(ZIE) 

Uwezo Tanzania National Curriculum Secondary School 
Development Centre (NCDC) Teacher 


Tanzania Institute of Education 


Educate! * 


Primary School 


(TIE) Teacher 

National Examination Council of | Girls to Lead Africa’ Kenyatta University 
Tanzania (NECTA) 

Organisation for Community Uwezo Uganda’ Kenya National 
Development (OCODE)* Examinations Council 


(KNEC) 


Global E-Schools and Community 


Foundation for Inclusive 


Kenya Institute of 


Initiative (GESCI * Community Help (FICH)* Curriculum 
Development (KICD) 
In Depth Consulting* Luigi Giussani Institute of Higher | Strathmore University 
Education (LGIHE)* 
Girls Leadership and Mentorship Kenya National Theatre 
Initiative (GLAMI)* 
Tamasha’? Zizi Afrique 


Archbishop Mihayo University 
College of Tabora (AMUCTA) 


University of Dar es Salaam 


Milele Zanzibar Foundation? 


National advisory committees 


MoEVT - Zanzibar 


Uganda National Education 


Teacher Service 


Board (UNEB) Commission (TSC) 

Tanzania Institute of Education Kyambogo University Kenya National 
Examinations Council 
(KNEC) 

National Parliament (MP) NCDC Kenya Institute of 
Curriculum 
Development (KICD) 

Ministry of Information, Culture, | Uganda National Teachers Union | PAL Network 

Youth and Sports (UNATU) 

MoEVT - Education Secondary School Headteachers _| Jaslika 


Association 


Student — Feza School 


Ministry of Education and Sports 


Kenya Primary School 


(MoES) — Private Schools and Headteachers 
Institutions Department Association 
MoEVT - Inspectorate UNATCO - Secretary General’s | Kenya Secondary 
Department Office School Heads 
Association 


(continued) 
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Table 11.2 (continued) 


Tanzania Uganda Kenya 
MoEVT - Inclusive Education and | MoES — Commissioner Zizi Afrique 
Life Skills Secondary Schools 


Prime Minister’s Office (PMO) -— | MoES - Directorate Education RELI Secretariat 
Labour, Youth, Employment and | Standards 
_People with Disabilities 


National Examination Council of 
Tanzania (NECTA) 


Zanzibar Examinations Council 
(ZEC) 


Zanzibar Teachers Union (ZATU) 
Uwezo Tanzania 


Femina 


Forum for African Women 
Educationalists (FAWE) Zanzibar 


University of Dar es Salaam 


MoEVT — Permanent Secretaries 
Office 


‘Denotes RELI Members involved in ALIVE 


11.3.2 Global Collaboration 


A central assumption for the ALIVE endeavour has been the autonomy to determine 
what gets measured and what the assessment data is used for within East Africa. 
This stands against a historical background in which how constructs are defined and 
tools developed has been dominated by research from the Global North. For this 
ambition to be achieved it was essential that the locus of control remained in East 
Africa. This, however, did not mean that ALIVE did not look to benefit from col- 
laborations and exchange of ideas and experience with individuals and organisa- 
tions globally. 

This global collaboration can be broken down into three kinds of collaboration. 
Firstly, ALIVE collaborated with a community of funding organisations. These 
organisations were central in investing in the vision of ALIVE, providing not only 
funding but also guidance and networking to support the delivery of ALIVE. Secondly, 
ALIVE collaborated with several individuals and organisations who provided tech- 
nical support or external facilitation on key activities such as the contextualisation 
research, tool development, and the learning process. Finally, ALIVE collaborated 
with the global education research community through conferences and 
publications. 
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11.3.2.1 How Did ALIVE Embody the Subskills of ALIVE as Defined by 
the Contextualisation Process? 


ALiVE encompassed all of the elements that were identified in the operational defi- 
nition of collaboration provided earlier in this chapter, throughout the contextualisa- 
tion, tool development and assessment processes. 

The issue of common task and shared goals are clear. Reflections from the learn- 
ing process place strong emphasis on the collaboration being goal focused, with a 
clear end point of having collected, analysed and published the data from the assess- 
ment of lifeskills and values. The main focus for collaboration was in shared tasks 
related to this goal. 

Looking at the traits of collaboration put forward by this definition, we see first 
negotiation throughout the process. The essential task of developing the assessment 
tools was to decide what should be measured and how, based on the outputs of the 
contextualisation studies. Participants in the reflection process frame this as a pro- 
cess of negotiation between different technical perspectives on the same challenge, 
for example between practitioners and academics, or those whose background is in 
lifeskills education, and those whose background is in statistics and research and so 
on. This negotiation was seen as an essential consequence of the diversity within 
technical teams. This process also accounts for the consensus building element of 
the definition. 

The collaboration in ALIVE stretched across borders and involved more than 20 
separate organisations. This required both interactivity and interdependence. The 
former was structured through regular meetings within and between national teams. 
The more than 40 individuals who comprised the three technical teams worked 
interactively on all tasks throughout the process. This was facilitated by technology, 
which became particularly important in the context of travel restrictions during the 
COVID-19 pandemic. This interactivity reflected the interdependence between 
individuals and teams. The diversity of the ALiVE collaboration was its strength. 
By having a diverse array of expertise, ALIVE could draw on a wide range of per- 
spectives (see Table 11.2 for affiliations of technical team members). The final 
result depended on interdependence, since no one person held all of the skills 
needed to complete the tasks. 

The ALiVE leadership structure centred around the co-Principal Investigators 
and the regional leadership team. This team was responsible for the overall agenda 
setting for ALIVE, as well as for building a culture of collaboration. The two main 
approaches used by leadership to create collaboration were structural, that is 
approaches which formally mandated time and resources for collaborative activi- 
ties, and cultural, that is the active promotion and modelling of collaborative 
approaches to working. 

The final component of ALiVE’s definition of collaboration is division of labour. 
While there was some division of tasks in ALIVE, it is not an element which 
characterises the program. In comparison to other similar programs, ALIVE priori- 
tised working together rather than allocating tasks to individuals or small teams. 
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Another way of looking at the concept of division of labour is to think of it as equal 
division of responsibilities between all participating organisations, rather than con- 
solidating activities into smaller sub-teams with specific expertise. In this latter way 
of thinking about division of labour, ALIVE fits the definition more closely. 


11.3.2.2 What Were the Perceived Benefits of Collaboration in Achieving 
ALiVE?’s Goals? 


The reflective data show how participants see the role of collaboration in the ALIVE 
effort. The ALiVE goals were to (i) develop contextualised assessments, (ii) use 
assessments for evidence led advocacy, and (iii) to develop a learning community 
within the program. 

For the first goal, collaboration was seen to be central in ensuring the contextual 
relevance of the constructs and tools. The process of working closely with adoles- 
cents, communities and government on the contextualisation process created new 
perspectives on what is important to measure. Working with adolescents to contex- 
tualise construct definitions forced the technical teams to confront and un-learn 
biases in how they perceived adolescence, life skills, and values. 

The collaborative approach taken by ALiVE was found to have a strong practical 
benefit in creating an enabling environment for the program’s advocacy work. 
National advisory committees involving government agencies and teachers was a 
feature of ALIVE from the beginning. Participants reflected that this allows for a 
greater ownership of the results of the assessment than could have been achieved 
otherwise. By providing government partners with the opportunity to ‘get their 
hands dirty’ and become involved with the assessment process from start to finish, 
it ensured that when results emerged, they were seen as legitimate and credible. 

The potential for this advocacy work was also reflected in engagement with com- 
munities during the assessment process. This process provided an opportunity to 
engage with families on the importance of life skills and values. These opportunities 
were infrequent, but represent a potential opportunity afforded by the collaborative 
approach. 


I was with a group of parents in the village in a collecting data about their understanding of 
life skills and values. It was a great concern for parents more than assessment. I realised that 
It was important to nurture life skills and values and that parents played a major role. ... 
Parents expressed the urgency of the need for support on how to nurture life skills and val- 
ues, more than on how to assess them. The need was more than I knew before. (Learning 
Insight on Community Engagement in Tanzania) 


Finally, the collaboration within the program was seen as central to the work on 
creating a learning community in ALIVE. Reflections emphasised the importance 
of diversity of opinions and experiences in creating an environment for organisa- 
tional growth within ALiVE. This commitment to learning as a collaborative activ- 
ity was manifested in the reflective process itself from which these findings 
are drawn. 
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11.3.2.3 What Was Learned About How to Put Collaboration 
into Practice? 


What Worked for Building Collaboration in ALiVE? Creating a collaborative and 
productive space within a collective of diverse individuals and organisations is not 
straight forward. Synthesising reflections from the first 2 years of the ALIVE pro- 
gram, three characteristics appeared central to collaboration; alignment of goals, 
principles and values, and leadership. 


The first of these emphasises the importance of understanding alignment between 
programmatic, individual, and organisational goals. As no one organisation was 
fully dedicated to ALIVE, this alignment ensured that individuals and organisations 
would continue to prioritise ALIVE activities, alongside their other commitments. 
Establishing alignment starts with a shared vision and understanding of the purpose 
of what is being done. Secondly, it involved understanding what each individuals’ 
and organisations’ goals are and how the program will support them in achieving 
those goals. If each collaborating individual or organisation has a clear view of these 
two things, it was felt that they would stay committed to the collaboration. 

During the reflection process, the importance of principles and values as foun- 
dations for collaboration emerged. The key principles and values that emerged 
were trust, transparency, and passion. Trust was defined as being important for 
establishing confidence in the fact that your collaborators share the same vision as 
you. This allowed individuals to seek and accept compromise. Transparency as a 
principle was important for establishing trust, as well as for ensuring that expecta- 
tions are clear. Passion underpins these, with teams reflecting that it was a shared 
goal, and the passion for seeing that goal actualised which drove forward 
collaboration. 

There was consensus that a program such as ALiVE, delivered by multiple 
organisations across multiple countries, requires strong leadership. The purpose of 
this leadership is not just to ensure that activities take place, but that they take place 
collaboratively. The reflections indicate that leadership that promotes collaboration 
has two key characteristics. The first is to lead by example when it comes to the 
values stated above, ensuring that leadership is based on trust, transparency, and 
passion of the shared goals. It also means placing importance on collaboration. 
Looking back at the first phase of ALIVE, it was felt that often when a decision was 
to be made between the ‘quick’ way and the collaborative way to complete an activ- 
ity, the leadership of ALIVE prioritised collaboration. 


What Challenges Were Faced in Building Collaboration? How Were They 
Overcome? With the technical and advisory teams contributing to the project, 
most individuals also had full-time workloads that demanded their attention. This 
situation led to long hours for members of the technical teams as they sought to 
find balance and ensure the project was successful. Individuals’ motivation and 
productivity can be affected by their perceptions of the contributions and participa- 
tion of other group members, which creates more challenges the larger the collabo- 
rating team grows. 
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Beyond the practicalities of working in teams of individuals, reaching compro- 
mise on deeply held beliefs presented challenges. This was particularly evident 
throughout the process of defining constructs and deciding how they would be mea- 
sured. The familiarity of team members with the competencies made it challenging 
to move beyond individuals’ preconceptions to find a shared conception of what 
should be measured and how. This led to decision-making stalemates which caused 
delays and frustration. However, in teams’ reflections it was felt that this slow road 
to consensus was an important process for addressing biases and moving beyond 
pre-conceptions. As outlined previously, principles and values such as trust, trans- 
parency, and passion were important in facilitating this consensus building. 

Beyond the universal challenges associated with building collaboration between 
heterogenous individuals and groups, ALIVE faced practical, systemic challenges. 
A significant part of the ALIVE process took place when COVID-19 had major 
impact through lockdowns and restricted movement locally and globally. Physical 
meetings were scarce. Abiding by global precautions and restrictions, the natural 
order and work medium for workshops had to be redefined. This called for innova- 
tion, and the use of more virtual communications than would otherwise have been 
the case. 

The crucial role that digital technologies play in human connections has been 
increasingly apparent as a consequence of the COVID-19 outbreak. Taking into 
consideration that ALIVE was a regional initiative, technology enabled the progress 
made by ALIVE. The various platforms that allowed for online meetings and work- 
shops, the highly essential breakout rooms, the shared documents platforms which 
facilitated simultaneous editing, the shared drives which anyone could access at any 
time — all of these were both the tools for doing and the tools for communicating. 


11.4 Discussion and Conclusion 


Findings from the reflective study do not deviate from the definition of collabora- 
tion arrived at through the contextualisation process. The reflections presented in 
this chapter show a strong practical manifestation of the subskills identified by the 
ALIVE definition (negotiation, consensus building, interactivity, interdependence, 
leadership, and accommodating each other). This provides a helpful starting point 
for other organisations embarking on their own collaborative endeavours. 

There is alignment between the findings from the reflection study and the frame- 
work presented by Nansubuga et al. (2023). The motivators which emerged most 
strongly were a clear vision and leadership. This focus on leadership may at first 
seem counter-intuitive, but the findings of this study reflect that in a complex pro- 
gram delivered by disparate and geographically separated organisations, a degree of 
centralised leadership is necessary. What is important is that this leadership priori- 
tises collaboration. This reflects a saying used often by ALIVE ‘if you want to go 
fast, go alone. If you want to go far, go together’. It shows that collaboration is not 
an automatic product of shared goals and principles, but something that takes 
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continuous work and leadership. There is also alignment in terms of the challenges 
derived through this reflection study and those identified by Nansubuga et al. (2023), 
with interdependence and the need to unlearn and re-learn through negotiation com- 
ing out in this study. 

The findings of this study were intended to reflect on what has been learned 
through the ALIVE program. The data that support the findings were co-created by 
the ALiVE team members who took part in the data collection and analysis process. 
The findings that emerge closely reflect therefore the experiences of those involved, 
as well as their intentions and visions for the future. This is a process that in no way 
tries to provide a neutral, detached view of the collaborative processes of ALiVE, 
but rather a reflection by team members on their perceptions of successes and the 
lessons learned along the way. 

The reflections from this learning exercise show the importance of collaboration 
across ALiVE’s three programmatic goals. Collaboration was important for creating 
learning community, but working collaboratively also enabled the development of 
tools and ALiVE’s work on advocacy. This shows the potential value of collabora- 
tive action in assessment development initiatives and can be a starting point for 
other organisations to think about how to make their work on assessment more 
collaborative. 
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Chapter 12 A 
Implications of ALIVE Process gest 
and Evidence for Policy and Practice 

in East Africa 


John Kabutha Mugo 


Abstract The process and product of the Assessment of Life Skills and Values in 
East Africa (ALIVE), including the opportunities created by this assessment, pro- 
vide a useful lens for viewing the mainstreaming of assessment of these compe- 
tences in the region. This chapter summarises ALiVE and provides perspectives 
on the use and lack of use of education evidence. Key insights cover the breadth of 
factors to be considered in influencing uptake by government. The chapter reviews 
the global and regional landscapes as they relate to life skills and values, and con- 
cludes that these competencies are increasingly acknowledged and adopted. 
Lastly, the chapter draws from the discussions in this volume to identify and discuss 
five implications for policy and practice. A call to bolden the focus and contextu- 
alise holistic learning is first made. Second is the call for a paradigm shift in assess- 
ment, including the review of assessment frameworks, leveraging technology, and 
prioritising the use of the evidence generated by the assessments. Third is the call to 
focus on teacher capacities, while considering that these would be facilitated by 
system-wide focus on capacity development. Fourth is the call for shared space and 
responsibility between parents and teachers, to perceive assessment as a common 
project between communities and schools. Lastly is the call for a resilient and com- 
prehensive approach that secures alignment, timeliness, engagement and 
institutionalisation. 


12.1 Introduction 


The chapters in this volume have introduced the ALiVE initiative, as both process 
and product. This social transformation program was designed to: generate evi- 
dence on life skills and values in the context of East Africa in order to develop and 
provide an open-source contextualised measurement tool; use the evidence to 
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inform policy and drive public conversations; and use tool development as a process 
for learning and self-empowerment for local experts in East Africa. Four achieve- 
ments stand out. 

First, a ground-up ethnographic study was conducted in 15 districts across 
Kenya, Tanzania, and Uganda. It was used as the basis for establishing local under- 
standings of selected competences. Second, a contextualised tool was developed to 
assess collaboration, problem solving, respect and self-awareness; and a large-scale, 
household-based assessment was conducted across the three countries, involving a 
total of 45,442 adolescents, comprising 22,092 males and 23,264 females, and 86 
not specified, aged between 13 and 17 years. Beyond the results, this assessment has 
contributed to both contextualisation and East Africa’s agency in contributing to 
global assessment in this emerging field. Third, four assessment reports and a sum- 
mary regional report were launched (to include separate reporting for Zanzibar, 
Tanzania) with in-person participation of more than 500 national-level education 
actors, and media activation (including community radio) that reached millions of 
East Africans. This was followed by sub-country level launches in more than 50 
districts, involving more than 3000 local level leaders and more than 10,000 head 
teachers. Fourth, 47 local experts were qualified via a learn-through-doing process 
that lasted 45 weeks between April 2021 and February 2022. Through this process, 
the ALIVE Academy has been established as the process that now sustains capacity 
development for assessment of twenty-first century skills in the region. 

The preceding chapters in this book have articulated these achievements at length 
and shared the emergent knowledge from the ALIVE process. However, before one 
can draw the implications for policy and practice from the evidence, it is necessary to 
clarify the conditions necessary for that evidence to inform policy in context. Without 
this clarity, policy recommendations merely constitute a tick-the-box exercise with 
little impact. 

Globally, there is general acceptance of and movement towards evidence-based 
policy making. Banks (2009) argues that policies that are not informed by evidence 
fall prey to the “law of unintended consequences” (p. 2), which often leads to costly 
mistakes. Head (2015) argues that the movement for evidence-use in policy devel- 
oped momentum in the 1970s, driven by Weiss (1979), Aaron (1978), Bulmer (1982, 
1987) and Rivlin (1971), and has yielded tremendous gains to public administration 
over the decades. 

Evidence-use in policy has met with equal measures of optimism and pessimism. 
First, there is acknowledgement that while existing documentation confirms that 
evidence does inform and even influence policy, the path is not linear. The conver- 
gence of evidence and policy is not just dependent on science but is highly contin- 
gent on political persuasion and on a variety of other economic and organisational 
factors (Head, 2015; Elliot & Popay, 2000). Second, there is little clarity on what 
works best to achieve evidence-informed decision making, considering factors like 
the capacities of policy makers in using evidence and conditions that promote or 
inhibit evidence use (Head et al., 2014; Stewart, 2015). 
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Head (2015) submits that even when sound evidence is at hand, the decision- 
making process in public administration is extremely complicated, and generally 
unfavourable to science-driven perspectives. He argues that policy making is driven 
more by political interests, conflicts, trade-offs, and compromises than by scientific 
rigour; and that “the policy process is inescapably anchored in politicised context 
and that some kinds of evidence are inevitably seen as more relevant than others for 
underpinning policy positions” (p. 6). This perspective supports earlier arguments 
(e.g., Maynard, 2006; Weiss, 1979) that evidence uptake is not just dependent on the 
robustness and clarity on what works but is mostly subjected to personal and politi- 
cal persuasions. In addition, governments prefer using their own internal evidence 
over that generated by external parties (Head, 2015), and are averse to evidence that 
presents an issue that is either new or presents an ideological dispute (British 
Academy, 2008; Head, 2010). 

Stewart (2015) adopts the proposal made by the Department for International 
Development (DFID, 2011) that offered a framework of four barriers to evidence 
use. First are the individual barriers faced by people in policy-making positions. 
These include lack of experience and capacity to assess evidence and build mutual 
trust, as well as negative attitudes towards change in general and evidence in par- 
ticular. Second are organisational barriers in the responsible institutions, which 
include unsupportive culture, competing interests, censorship, and mistrust of exter- 
nal evidence, as well as other barriers like cultural and religious notions. Third are 
communication and relationship barriers, which include message and messenger 
choice, information overload, use of academic jargon and unactionable messages. 
Lastly are timing barriers, including the lack of time to understand and reason 
together, and lack of alignment between researchers’ and decision makers’ time- 
frames and priorities of the moment. 

A further consideration is how researchers package and communicate evidence. 
Many researchers naively assume that rigorous evidence and good communication 
are the sole drivers of evidence-use in policy making (Cherney & Head, 2011), 
while others make recommendations that are both politically and economically 
untenable (Stewart, 2015). In confronting these barriers, Head (2015) suggests that 
any evidence-use case must provide a framework that balances the persuading of 
decision-makers to formulate a policy, the support needed to roll out policy imple- 
mentation, and the provision of the necessary conditions for implementation. 
Further to this, Strydom et al. (2010) suggest that researchers must consider organ- 
isational factors (like structure and function), socio-economic contexts (affordabil- 
ity), communication factors (message attributes), notions of credibility of both the 
evidence and researcher, as well as the political priorities of the reigning govern- 
ment (Elliot & Popay, 2000). 

Head (2015) proposes the concept of knowledge brokering. This concept calls 
for knowledge sharing and simplification of information to facilitate communica- 
tion between evidence producers and users, and approaches that include face-face 
exchanges, formation of networks and communities of practice, and facilitated 
sessions. 
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These global perspectives bequeath to ALIVE a useful framework for viewing 
the implications of the ALIVE process and evidence for policy makers in East 
Africa. This framework includes first the aspect of alignment, how the evidence is 
packaged and communicated to speak to the economic contexts and political inter- 
ests of the three countries. Second is the aspect of timeliness. The curricular revi- 
sions made by East African systems confirm that life skills and values are both a 
current point of focus and non-controversial for all four education jurisdictions 
(including Zanzibar as separate from Tanzania mainland). Third is the aspect of 
structured engagement. This points to the need to go beyond suggestions for 
evidence-uptake, and factor in support for implementation of those suggestions. 
This aspect includes the capacities and brokering needed to make policy making 
and implementation a reality, driven by trust and careful identification of messen- 
gers. Lastly is the dimension of institutionalisation. The recommendations must 
include a way to sustain the two-way relationship between the ALIVE community 
and decision-makers in government, in a form that is aligned to the needs, priorities 
and capacities of both groups. 


12.1.1 Global Education Policy Pertaining to Life Skills 
and Values 


In the initial chapters of this volume, Chaps. | and 3, Care (2024) and Giazomazzi 
(2024) conclude that global policy has embraced the inclusion of twenty-first cen- 
tury competences in education. The Sustainable Development Goal (SDG 4.7; 
UNESCO, 2015) makes no explicit mention of these competences, but talks more 
broadly about the knowledge and skills needed to promote sustainable develop- 
ment. Though the reporting on this target is most recently biased towards climate 
change, a few initiatives continue to drive SDG 4.7 as encompassing social emo- 
tional learning and related competences. 

The SDG 4.7 was preceded by the Learning Metrics Task Force (LMTF), which 
led global consultations to explore the improvement of learning outcomes as well as 
the skills needed for twenty-first century learning, working, and living. The LMTF 
covered a total of 118 countries (11 in Africa) and concluded that besides the usual 
academic subjects, learning should include students’ physical, social, and emotional 
wellbeing, as well as approaches to learning how to learn (Anderson & Ditmore, 
2016), described in some detail by Giazomazzi (2024; Chap. 3, this volume). 
However, more recent publications have expressed the perspective that progress is 
slow, and that universal integration of these competences may not be achieved by 
2030, unless the pace is hastened (Jukes et al., 2018; Care et al., 2018). 

Several mechanisms support the monitoring and implementation of SDG 4.7, 
and some focus on partnerships in sub-Saharan Africa. These include the Networking 
to Integrate SDG 4.7 and Social and Emotional Learning into Educational Materials 
(NISSEM), the Inter-agency Network for Education in Emergencies (INEE) and 
Harvard’s Ecological Approaches to Socio-emotional Learning (EASEL) 
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Laboratory, the Karanga Global Alliance, the Global Learning Community coordi- 
nated at Utrecht University, and the Global Centre for the Development of the 
Whole Child at the University of Notre Dame (USA). The OECD has also focused 
on social-emotional learning, and has developed and conducted several assess- 
ments, including pilots with selected countries in sub-Saharan Africa. 


12.1.2 Policy Landscape in East Africa 


In East Africa, life skills and values are not new in education policy. There have 
been multiple curricular revisions over the past two decades. The inclusion of these 
constructs has been associated with the movement of the countries to competency- 
based curricula, commencing in 2005 in Tanzania, 2016 in Kenya, and 
2019 in Uganda. 


12.1.2.1 Kenya 


The Competency-Based Curriculum (CBC) commenced implementation in 2016 
and reached Junior Secondary (Grade 7) in January 2023. The CBC prioritizes 
seven core competences for basic education, as well as eight values that education 
should develop (Kenya Institute of Curriculum Development, 2017; Heto et al., 
2020). The mode of delivery clarifies integration in academic subjects, with the cur- 
riculum clearly identifying the competences and values that the teacher should 
develop within each topic. Teacher training has included aspects of these compe- 
tences, though little documentation exists on related classroom practice. Learning 
assessments have adopted a competency measurement approach, relying heavily on 
use of rubrics and portfolios in formative assessment. The learner assessments at 
Grades 3 and 6 have included some of these competences at classroom level, though 
these have not been released publicly with other learning areas. There is no shared 
evidence from summative assessment, and no evidence on these competences in 
policy and public debate. 

Wafubwa and Csikos (2021) observe that despite the clear definition of these 
competences in the curriculum, the assessment framework is too sketchy to direct 
teachers how to observe and accurately capture competence in demonstrated behav- 
iour. Originally, life skills had been adopted as a stand-alone learning area for junior 
secondary, but this was later axed by the Presidential Working Party in 2023, seek- 
ing to reduce learning areas for junior secondary. 


12.1.2.2 Tanzania Mainland 


While Tanzania is one country, it is important to be aware that basic education is a 
devolved function for Zanzibar. 
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Life skills education is not new in Tanzania. In the mid-1990s, life skills was a 
subject taught in all schools and covered social skills such as citizenship, emotional 
and behavioural competencies including acceptable behaviour and respect, as well 
as some cognitive competencies such as critical thinking, among others (Care, 
2024; Chap. 1, this volume). In 2005, Tanzania initiated the movement to 
competency-based curriculum through a major reform, which also saw the removal 
of life skills from the list of examinable subjects. Giacomazzi (2024; Chap. 3, this 
volume) documents that while this was well-intended, it led to rapid de-prioritisation 
of life skills by the teachers, and the time allocated to life skills was often used to 
teach examinable subjects. In 2019, the government made a curriculum revision, 
and with support from UNICEF, revised the National Life Skills Education 
Framework, including the development of a detailed strategy and formative assess- 
ment tools for primary and secondary schools. There is documentation that despite 
these changes, the implementation of life skills and values remains problematic, and 
the teaching and learning processes may not be producing the targeted competen- 
cies (Nkya et al., 2021). 


12.1.2.3 Zanzibar 


A national life skills framework was developed in 2010. The framework was 
intended to promote quality provision of life skills education, with the focus “to 
attain positive behavioural changes among children and young people in Zanzibar 
for their own individual and societal benefits in relation to preventing HIV/AIDS 
infections, substance abuse, early pregnancies, gender based violence and lead 
more positive and productive lives” (Zanzibar Ministry of Social Welfare, Youth 
Women and Children Development, 2010, p. 8). In 2022, the President of Zanzibar 
appointed a taskforce to advise on reforming education. The taskforce report rec- 
ommends, inter alia, proper definition and integration of life skills and related 
competences in the curriculum, teacher training and classroom practice and pro- 
ducing prioritized life skills and values through education. The changes are 
expected to be implemented from 2023. The government, recognizing the insuf- 
ficient emphasis on life skills, has prioritized the refocus and review of the cur- 
riculum. Efforts are being made to integrate life skills into early childhood 
development and foster collaboration with ALIVE to incorporate these essential 
skills and values into the curricula. The district-level ALIVE launches have rein- 
forced the government’s commitment, emphasizing the importance of political 
leaders incorporating assessment data into their planning of national plans and 
budgets (Issa, 2023). These actions reflect the government’s dedication to address 
policy shortcomings and ensure that decision-making processes are grounded in 
relevant data. 
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12.1.2.4 Uganda 


In Uganda, life skills were embedded in the 2019 Sexuality Education Framework, 
which identified 20 explicit competencies, including conflict resolution, decision 
making, communication, negotiation, leadership, goal-setting, assertiveness, refusal 
skills, coping with emotions and stress, self-awareness, self-control, self-esteem, 
help- and health-seeking behaviours, time management, employability skills, ana- 
lysing the media, and journaling. However, the focus on sexuality education received 
pushback from religious bodies, which operate about three quarters of the schools 
in Uganda. The Ministry’s response was a later revision that broadened the curricu- 
lum beyond sexuality education. 

In 2019, the National Curriculum Development Centre (NCDC) reformed the 
lower secondary education curriculum and prioritised a number of competencies. 
These were referred to as generic skills, and included critical thinking and problem 
solving, creativity and innovation, communication, co-operation and self-directed 
learning, mathematical computation, and proficiency in ICT (NCDC, 2020). The 
curriculum also included life skills as a learning area under cross-cutting issues, 
which included decision-making, stress management and emotional regulation, and 
to “solve problems met in different situations” (p. 10). The assessment framework 
for the new curriculum was released in year 2022 and it requires the teachers to 
specify the level of students’ achievements in terms of subject-specific competen- 
cies and generic skills. Further to this, a new curriculum framework was drafted in 
2023 for Upper Secondary which includes prioritized values. 


12.2 Five Implications for Policy and Practice 


The ALiVE results have woken us from slumber. We had nothing to show us where we were. 
Now, the call to action is loud (Dr Grace Baguma, Head of NCDC in Uganda, at the launch 
of the ALIVE report on December 13, 2022). 


The ALIVE contextualisation study, assessment processes and results, have impli- 
cations for policy formulation and implementation across the three countries. The 
evidence backing these implications draws from three main sources, two of which 
are covered extensively in chapters in this volume: ALiVE contextualisation studies 
(Shariff et al., 2024; Giacomazzi, 2024; Chaps. 2 and 3); and ALiVE assessment 
process and results (Nansubuga et al., 2024; Nakabugo et al., 2024; Mutweleli et al., 
2024; Ariapa et al., 2024, Chaps. 7, 8, 9, and 10). The third source derives from 
policy conversations during the report launches in Zanzibar (November 17, 2022), 
Kenya (November 25, 2022), Uganda (December 13, 2022) and Mainland Tanzania 
(January 26, 2023). The admissibility of policy conversations as evidence draws 
confidence from the definition by Strydom et al. (2010), which views evidence as 
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both research-derived and anecdotal, encompassing knowledge from experts as well 
as lay persons. The process and results of ALIVE yield the conclusion that while 
there has been consistent progress made in embedding life skills and values into 
policy, curriculum and classroom practice, the intended learning outcomes are 
not yet realized for a majority of the learners. At the same time, this is recognised 
by the systems in their efforts to more clearly embed the competencies in curricu- 
lum and assessment, as well as teacher training and support. Overall, five implica- 
tions have emerged which speak to both policy and practice. 


12.2.1 Implication 1: Boldening the Focus and Fully 
Contextualizing Holistic Learning 


The importance of broadening the focus of foundational learning beyond literacy 
and numeracy, has been well articulated in this book. Care (2024; Chap. 1, this vol- 
ume) refers to the World Bank’s conceptualisation of ‘learning poverty’ as referring 
solely to reading and numeracy. This minimalist approach to foundational learning 
is creating a push on low-income economies to sacrifice the holistic development of 
the child, and this abandoning of critical competencies in the nascent stage of the 
child’s development may have dire effect not only on their development and attain- 
ment of full potential, but also on the competitiveness of nations in producing skills 
for the changing world. 

The results of the ALiVE assessments (Ariapa et al., 2024; Chap. 10, this vol- 
ume) have demonstrated that while the three East African countries have been on a 
journey to include life skills and values in the curricula for now close to two decades, 
the interventions may not yet be delivering the intended outcomes. In brief, the 
ALIVE results have shown that most adolescents have commenced their journey to 
skill acquisition and are at varied points. For instance, in problem solving, most 
adolescents can identify the problem, but fewer are able to gather and present infor- 
mation needed to characterise the problem, and most of them identify only one 
solution to a problem, even when they are clearly prompted to identify several. 
Similarly, in terms of respect for others, most adolescents interpret bad behaviour as 
a lack of respect for others or self, with fewer taking conciliatory steps to resolve 
situations, and only a few being aware of links between respect for property and 
respect for persons. 


12.2.1.1 Bold Focus 


The dissonance between what happens at school (curriculum) and out in society 
(daily lives) (Care, 2024; Chap. 1, this volume) is an impediment to curriculum 
implementation. In other words, a focus on curriculum and school only, with no 
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associated focus on the influence of society, may not work. A bold focus is needed, 
in which three things happen. First, the society recognises the need for change and 
embraces the direction of change. Second, curricula and schooling take up this soci- 
ety challenge. Third, consistency between society and school and curriculum is 
achieved through increased awareness of communities, adapted political leadership 
and appropriate investment to support the new direction of education. 

This focus is needed to identify and articulate the competencies that each country 
prioritises, and clearly define and articulate these in the curriculum. The approach 
taken by ALiVE in carefully inter-weaving skill definitions and skill structures 
offers one option for curriculum experts in the region. Second is the pursuit of 
authentic assessments, which focus not on reporting scores, but ways of capturing 
the competencies and values as they are demonstrated in real life. Again, the ALIVE 
assessment has demonstrated that measurement of these competencies is possible 
and challenges the notion that life skills and values are impossible or difficult to 
measure. Third is equipping teachers with the pedagogical strategies needed for 
enhancing life skills, beyond the academic-only curriculum. Lastly is the engage- 
ment of society. Political leadership, together with community leaders, must cham- 
pion this so that communities and parents not only change the way children are 
nurtured and educated, but are active in efforts to achieve coherence with the cur- 
riculum and address any dissonance. 


12.2.1.2 Contextualisation 


Contextualisation and authenticity are key components, not only for developing 
appropriate assessment strategies, but also providing the space for children to dem- 
onstrate these competences in real life (Care, 2024; Chap. 1, this volume). It is 
important that as countries embolden their focus on life skills and values, they also 
ensure that the prioritised competences are well aligned to the local context 
(Mutweleli et al., 2024; Chap. 9, this volume) and not just pick from other contexts 
where these may be well documented. 

In a related publication from this work, Giacomazzi et al. (2022) make a strong 
case for contextualisation, not just for assessments, but also for pedagogical 
approaches. Giacomazzi (2024; Chap. 3, this volume) has presented the results of 
the ALIVE contextualisation study conducted in 15 districts across East Africa. 
These results guided and framed the development of assessments of problem solv- 
ing, self-awareness and respect, and collaboration (Care & Giacomazzi, 2024; 
Chap. 4, this volume; Ngina et al., 2024; Chap. 5, this volume; Scoular & Otieno, 
2024; Chaps. 6, this volume), demonstrating the potential of contextualisation to 
inform curriculum and assessment. The ALiVE contextualisation unearthed 
unique skill dimensions not captured in global literature. The differences estab- 
lished highlight the importance of not striving for harmonisation across contexts 
but rather, focusing on common components as well as understanding the compe- 
tencies in their contexts. 
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12.2.2 Implication 2: A Paradigm Shift to Assess Life Skills 
and Values 


The way of assessing academic subjects in East Africa’s education system is 
fundamentally different to what is needed for measuring generic competencies. 
The sometimes non-visible nature of life skills and values, and their high depen- 
dency on context, necessitates a different paradigm (Nansubuga et al., 2024; Chap. 7, 
this volume; Mutweleli et al., 2024; Chap. 9, this volume; Nakabugo et al., 2024; 
Chap. 8, this volume; and Ariapa et al., 2024; Chaps. 10, this volume). Three impli- 
cations emerge from these chapters, and from the knowledge accumulated through 
the ALIVE assessment. 


12.2.2.1 Clear Shift of Assessment Frameworks 


Life skills and values are dynamic in nature and are less known and understood in 
the context of formal education. Accordingly, it is particularly important for the 
assessment of these competencies that measurement procedures commence with 
proper definition of the constructs, clarification of the constructs, development of 
tasks and items, collection and scoring of responses, through to the production of 
measures (Ariapa et al., 2024; Chap. 10, this volume). 

To integrate these assessments in the system, investment is necessary. Frameworks 
must be developed for each competency or value in order to ensure a standardised 
approach, and to ensure that the outcome will be true to context. The indulgence of the 
national assessment institutes to contextualise the varied approaches, together with a 
research component to establish their validity and reliability, is critical. From ALiVE’s 
experience, the development of such frameworks includes a systematic way of estab- 
lishing the assessable and non-assessable components of a skill, exploring a variety of 
options to arrive at the most suitable methods for both capturing the responses and 
behaviours, scoring and analysis, as well as achieving the highest possible psychomet- 
ric qualities for the assessment tasks. In all this, a framework must deliver a simple 
approach for teachers to use for formative purposes, as well as schools and the system 
to use for the summative purpose of monitoring learning outcomes. 


12.2.2.2 Leveraging Technology 


Two chapters in this volume, Chaps. 7 and 8, Nansubuga et al. (2024) and Nakabugo 
et al. (2024), have justified the need to consider use of technology in assessment, 
and used the ALIVE approach to illustrate use of digital assessment. The fact that 
the assessment of life skills and values is a new thing in East Africa offers the oppor- 
tunity to introduce it in a new way that does not require change management. The 
ALiVE assessment, which used smartphones (rather than tablets or other devices 
that might be more difficult to avail) and Kobo Toolbox (an open-sourced software, 
rather than software or applications that need purchase or subscription), offers 
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useful reflection on how the adoption of technology could revolutionise assessment 
and lower its cost. In a school setting, options might include the development and 
sharing of level-based item banks that teachers could draw from, the development 
of a classroom-based digital platform for recording observed behaviours and 
responses, and the leveraging of technology to automate developing and sharing 
reports. 


12.2.2.3 Inducing Use of Assessment Evidence 


Being a new area of education focus, the assessment of life skills and values offers 
the opportunity for advocacy, to ensure that stakeholders in the system will under- 
stand, embrace, and support successful mainstreaming. Thus, the suggestion by 
Nakabugo et al. (2024; Chap. 8, this volume) is worth consideration, that we treat 
the data generation process just as importantly as the data itself. This process 
needs to be open and inclusive (co-creation), facilitating the participation of as 
many stakeholders as possible, including policy makers and other key actors in 
the system like local officers and even parents. A second suggestion that emerges 
from ALiVE and articulated by Head (2015), is that the communication of assess- 
ment results needs to speak to the political process, and take advantage of critical 
political moments. One such moment is the summative assessment for the new 
lower secondary curriculum in Uganda planned for 2024, the education reform 
underway in Zanzibar, and the pronouncement of priorities of the current govern- 
ment in Kenya through the report of the Presidential Working Party on educa- 
tion reform. 


12.2.3 Implication 3: Teachers’ Assessment Capacities 


Care (2024; Chap. 1, this volume) argues that a shift in curriculum without a 
matched shift in teacher attitudes and capacities, and learning resources, may not 
lead us to where we want to go. The preference of the integrated approach by East 
African education systems has ushered in the need for teachers to not only pay 
attention to the achievement of lesson objectives for academic subjects, but also 
objectives linked to additional competencies. This is not easy to achieve, even for 
the best teachers. 

The conversation among policy actors at the ALIVE launch in Uganda arrived at 
the conclusion that while a mindset shift was needed, the acquisition of these com- 
petences (in both teachers and learners) necessitated a shift in the way of life, not 
merely a pedagogical one. This, it was seen, implied wide scale mobilisation of 
teachers to understand why these competencies are needed as a matter of priority, 
before we can talk about their capacities to teach or develop them. As one partici- 
pant indicated: A national approach is necessary, where we walk together as a 
nation. All teachers must be mobilised to understand, own and nurture life skills. 
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Related to this, a further perspective was noted, that while development of academic 
knowledge and skills is dependent on the content knowledge and pedagogical com- 
petence of the teachers, life skills and values require demonstration, implying that 
teachers would need to possess these. 

Teachers’ lack of assessment capacities has been documented (e.g., Giacomazzi 
et al., 2022). However, before we reach teachers, the upstream functions of national 
assessment and quality assurance systems need to accumulate expertise and pass 
this on to the trainers of teachers. As noted in various chapters in this volume, 
assessing these competencies requires specific growth in expertise. The ALIVE 
academy, which now builds expertise not through boardroom training, but through 
the measurement development process, offers a useful model for education systems 
in the region. Through this model, it would be possible not only to build a critical 
mass of experts over time, but also to harness cross-country collaborative energy for 
the region. This could produce shared item banks and repositories that could benefit 
everyone. Once expertise starts accumulating at this level, it will be possible to 
onboard teacher trainers in a systematic way and unlock the available resources for 
teacher training at pre- and in-service levels. This can go hand-in-hand with integra- 
tion of assessments at classroom and system levels, adopting an iterative improve- 
ment approach. 


12.2.44 Implication 4: Assessment as a Shared Space Between 
Teachers and Parents 


Recalled at the ALIVE launch event in Dar es salaam, the oft-repeated statement ‘it 
takes a village to bring up a child’, remains as true as ever. The conversation in 
Tanzania noted that the school was unnecessarily burdened by problems that fami- 
lies and communities should solve: ‘many children show up at school when they are 
already broken, and it becomes the impossible task of teachers to mend them’. 

A tough inference can be drawn from these conversations. Intervening in these 
life skills and values competencies needs communities to engage, where adults and 
parents are held accountable for demonstrating the skills and values to children. 
Second is the need for consistency between the home and school value systems, 
through dialogical relationships between parents and teachers in promoting ways of 
living that promote the nurturing of life skills and values. Third, the wider commu- 
nity and society, including the media and other communication platforms such as 
community and political meetings must be seen to be adopting the prioritised values. 

ALiVE has two processes that connected closely with communities and par- 
ents, and lessons could be drawn for this. First, the contextualisation exercise that 
was conducted over the COVID period was situated in communities, involving 
local leaders, teachers, parents, social and health workers, religious teachers (cat- 
echists, maalims), and other people who worked with adolescents. Besides pro- 
ducing useful insights for the understanding of the constructs in context, this 
process enabled community members to contribute to the assessment 
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development process. This contributes to the suggestions made severally in this 
volume (Care & Giacomazzi, 2024; Mutweleli et al., 2024; Ariapa et al., 2024, 
Chaps. 4, 9, and 10), that the assessment of these competences must mirror the 
daily lives of children in the community. 

Second, the ALIVE assessment was conducted in the household, offering oppor- 
tunity for parents to observe and ask questions on the assessment. The opportunities 
and challenges of household-based assessments are well covered in the chapter by 
Nakabugo et al. (2024). Most highlighted is the awareness raising that occurs 
through these assessments, and the immediacy of feedback obtained through this. 
However, one challenge that arises is how to balance the exclusion of teachers, 
when theirs is the accountability burden to nurture the competences. There are sug- 
gestions to flip this, where school-based assessments could consider open space for 
parents and other key actors to participate, as a shared exercise. For instance, the 
teacher could open space for parent representatives to participate, bringing their 
experience to it, but also drawing useful parenting lessons from it. 


12.2.5 Implication 5: A Resilient, Comprehensive Approach 
for Change 


Heated debate ensued at the Uganda ALIVE launch. One side of the conversa- 
tion argued that it did not make sense to measure something where no structured 
intervention exists in the education system, and the other side argued that evi- 
dence is needed to drive the shift. The participants recalled a three-decade his- 
tory of life skills education, from early 1990s, and how focus oscillated across 
epochs where these skills were and were not examined. Even while the construct 
of life skills has evolved over this time (Care, 2024; Chap. 1, in this volume), 
the point was clearly made. Without evidence, such as that provided by ALIVE 
at this launch, it may have been impossible to achieve such depth and intensity 
of conversation. 

This focus on three decades of intermittent life skills education is a clear indica- 
tion that change happens slowly, steadily. ALiVE’s participatory process of learning 
how to develop the assessment tool across more than 40 weeks, with 47 East 
Africans, is one clear manifestation of the resilience of the change. A reflection on the 
journey forward takes us back to the opening section of this chapter, to five factors. 


12.2.5.1 Achieving Alignment 


First there must be alignment between the call to integrate assessments of life skills 
and values on the one hand, and political priorities and economic realities on the 
other (Stewart, 2015). Proper packaging and communication of the evidence ensu- 
ing from assessments is necessary to achieve political persuasion (Elliot & Popay, 
2000) and the needed convergence with political interests of each country. One 
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lesson that ALIVE offers is the opening up of assessments as negotiated space with 
policy makers, rather than producing results, taking evidence to policy makers, and 
exposing this to the risk of doubt and rejection (Nakabugo et al., 2024; Chap. 8, this 
volume). Given that political interests and economic realities keep shifting, the 
assessment process must constantly be re-aligned to emerging curricula and 
pedagogies. 


12.2.5.2 Exploiting Timeliness 


Giacomazzi (2024; Chap. 3, this volume) and Shariff et al. (2024; Chap. 2, this 
volume) concur that the moment is right for the three countries to pay attention to 
the development of this field. Now, life skills and values are perceived as a non- 
controversial and timely debate. Within just 3 years, the ALIVE process has attracted 
immense attention, and now presents a significant opportunity to heighten aware- 
ness of the implementation consequences of recent policy. The strengthened bond 
between state and non-state actors provides a convincing example of how collabora- 
tive practice can shift discussion and stimulate action. 


12.2.5.3 Sustaining Engagement 


Keeping abreast with the political and economic shifts necessitates sustained 
engagement, beyond suggestion and through to oversight and implementation. The 
ALiVE process has led to structured partnerships with government agencies across 
the three countries, as the only pathway to realizing change at the classroom and 
child level. Three ingredients may be needed to sustain these engagements. First is 
cultivating trust and collaboration among government institutions (curriculum and 
assessment centres), so that they break the expert-silos that currently exist. Second 
is to promote partnership and community building among experts in and out of 
government. Third is the formalization of relationships and inventing cross- 
institutional collaboration, so that the sharing of available resources can be 
smoothed. To achieve this, the programme has rebranded to ‘Action for Life Skills 
and Values in East Africa’, recognising that assessment alone is cannot achieve the 
needed change, and expanding focus to support developments in curriculum, 
classroom-based assessment and pre-service teacher training. 


12.2.5.4 Institutionalising What We Produce 


Chapters in this volume have described some barriers to assessments of life skills 
and values. Beyond the technical difficulties faced in producing evidence, there are 
difficulties that concern evidence use. As mentioned, Stewart (2015) describes four 
key barriers to evidence use. Beyond the individual, communication and relation- 
ships, and timing, lies the organizational or institutional. The anchoring of 
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assessment processes and tools in existing institutions is key to change. System 
mapping approaches offer useful strategies for the embedding of capacities, pro- 
cesses, and tools. The myriad spaces for institutionalisation include the political 
processes of legislation, the technical processes of education strategy and policy 
development, the teacher training ecosystem, the quality assurance and school- 
based teacher support systems, as well as the many supportive networks that include 
teacher and parent associations and civil society networks. These are the institu- 
tional spaces in which sustained engagement must focus. 


12.3 Conclusion 


Since conclusion of the first phase of ALiVE, in what might in retrospect be seen as 
a deep learning about life skills and values and their assessment, the products of that 
phase and the discussions with the four education systems across the three countries 
have generated and contributed to continuing reform and development initiatives 
across Tanzania mainland, Zanzibar and Uganda. First, these initiatives are taking 
the form of increasing focus on prioritization, definitions and structures of skills — 
making these explicit in the curriculum. The second dimension is embedding these 
in assessments, testing the different approaches to assessment and building system 
capacities in this area. The third dimension of the initiatives is on teacher training, 
embedding the competencies in teacher training curricula and working with the 
teacher training institutions to develop training methods best suited for these skills. 
If this momentum is sustained, the collaborative work of experts in and out of gov- 
ernment is expected to accelerate the movement towards realisation of the learning 
outcomes across education levels. More valuable still, will be the contribution of 
East Africa to global knowledge, on what works and what does not in achieving 
system shift outcomes for learners’ life skills and values. 
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